CCD-Bench: Dual Benchmarking in LLMs & CCD Detectors

• CCD-Bench is a polysemous term referring to distinct benchmarking paradigms for evaluating large language model cultural decisions and charge-coupled device performance.
• The Culture-Conflict Decision Benchmark evaluates LLMs via 2,182 dilemmas across cultural domains, revealing pronounced biases and constrained rationale pluralism.
• CCD detector benchmarks employ both simulation and lab-based setups to achieve precise calibration, systematics characterization, and performance assessment.

Searching arXiv for "CCD-Bench" and closely related CCD benchmarking work. CCD-Bench is an ambiguous label used in two distinct research lineages on arXiv. In one lineage, “CCD-Bench” denotes the “Culture-Conflict Decision Benchmark,” a benchmark for evaluating LLM decision-making under explicit cross-cultural value conflict (Rahman et al., 3 Oct 2025). In another lineage, the same label is used descriptively for benchmarking infrastructures for charge-coupled device characterization, calibration, and systematics studies, spanning space photometry simulators, optical quantum-efficiency benches, X-ray detector test stands, photon-transfer analysis, and LSST-style laboratory systematics campaigns (Zima et al., 2010, Krishnamurthy et al., 2019, Bradshaw et al., 2018, Ruo-Berchera et al., 2014, Chattopadhyay et al., 2020, Astier et al., 2019, Bernardes et al., 2018, Bonano et al., 2023). The term therefore does not designate a single canonical artifact across the literature; rather, it refers either to a cultural-conflict benchmark for LLMs or to a family of CCD benchmarking frameworks for detector performance and calibration.

1. Terminological scope and disambiguation

The most explicit contemporary use of the name appears in “CCD-Bench: Probing Cultural Conflict in LLM Decision-Making,” where CCD-Bench is expanded as “Culture-Conflict Decision Benchmark” and is designed to evaluate how LLMs choose among culturally grounded, mutually legitimate response options under overt value conflict (Rahman et al., 3 Oct 2025). The benchmark targets decision-making rather than cultural recall, demographic value prediction, or single-axis bias scoring.

A separate usage emerges in instrumentation and detector-characterization literature, where “CCD-Bench” functions as a descriptive name for a benchmarking suite or test bench centered on charge-coupled devices. In this sense, the term is used for precision optical benches, X-ray detector stands, laboratory campaigns for systematics characterization, and end-to-end simulation environments that can be translated into benchmark scenarios and metrics (Krishnamurthy et al., 2019, Chattopadhyay et al., 2020, Bradshaw et al., 2018, Zima et al., 2010). This suggests that “CCD-Bench” is polysemous across fields, with one branch rooted in LLM evaluation and another in CCD detector metrology and calibration.

2. CCD-Bench as the Culture-Conflict Decision Benchmark

In the LLM literature, CCD-Bench comprises 2,182 open-ended dilemmas spanning seven domains: Arts, Education, Family, Wellness, Work, Scientific, and Lifestyle (Rahman et al., 3 Oct 2025). Each dilemma is paired with exactly ten anonymized response options mapped to the ten GLOBE cultural clusters representing 62 societies: Anglo, Latin Europe, Nordic Europe, Germanic Europe, Eastern Europe, Latin America, Sub-Saharan Africa, Middle East and North Africa, Southern Asia, and Confucian Asia. The benchmark is constructed to expose models to “value-plural dilemmas where several culturally grounded solutions are equally defensible,” then require the model to choose and justify one anonymized option.

Dataset construction follows a two-step pipeline in which a state-of-the-art reasoning model generates, for each cluster, a one-sentence answer that does not mention the culture and a one-sentence rationale that references cultural values. Similarity pruning uses pairwise MinHash Jaccard similarity over 3-gram shingles within each 10-tuple, with regeneration triggered when similarity exceeds $\tau = 0.70$ (Rahman et al., 3 Oct 2025). Across the final corpus, the pairwise MinHash Jaccard similarity among alternatives has mean $M = 0.123$ and standard deviation $\sigma = 0.109$ over $N = \binom{10}{2} \times 2{,}182$ comparisons. Human verification confirms that each option is reasonable within the target cluster, that no option names the cluster explicitly, and that options are semantically distinct.

To mitigate ordering effects, CCD-Bench uses a Stratified Latin Square design. For each domain separately, a $10 \times 10$ Latin square determines display order, so that each cluster appears exactly once in every ordinal slot and precedes or follows every other cluster exactly once across the corpus. The reported Latin-square balance score is approximately 99.8%, and ordering effects are negligible according to chi-squared tests and Cramér’s $V$, with all clusters yielding $V < 0.10$ (Rahman et al., 3 Oct 2025).

3. Evaluation protocol and findings in the LLM benchmark

The benchmark evaluates 17 leading non-reasoning LLMs available via OpenRouter, with “non-reasoning” defined as evaluation without explicit chain-of-thought prompting, tool use, or auxiliary retrieval (Rahman et al., 3 Oct 2025). For each dilemma, models see the ten anonymized options in Stratified Latin Square order and are asked to select one option and provide a justification. Aggregation proceeds by counting selections per cluster globally and per domain, using raw tallies without smoothing or normalization. The paper reports total valid selections of 36,977 and 37,092 rationales out of the theoretical maximum of 37,094.

The principal empirical result is a marked skew in cultural preference distributions. Across the 17 models, Nordic Europe is disproportionately preferred, with mean 20.2% of selections, while Germanic Europe averages 12.4%, Sub-Saharan Africa 11.5%, and Anglo 11.3% (Rahman et al., 3 Oct 2025). Eastern Europe at 5.6% and Middle East and North Africa at 5.8% are underrepresented, and no model exceeds 8.2% for either cluster. The paper further reports that Latin Europe shows the smallest coefficient of variation, $CV = 0.09$, while Anglo exhibits the largest, $CV = 0.23$.

Rationale analysis codes explicit mentions of GLOBE dimensions in model-generated justifications. Across $N = 37{,}092$ rationales, 87.9% reference two or more dimensions, which the paper interprets as formal pluralism (Rahman et al., 3 Oct 2025). However, dimension usage is highly concentrated: Performance Orientation is 16.2%, Future Orientation 15.1%, Institutional Collectivism 14.9%, In-Group Collectivism 13.5%, Humane Orientation 13.0%, Power Distance 12.5%, Uncertainty Avoidance 9.5%, Gender Egalitarianism 2.7%, and Assertiveness 2.6%. A single pairing, Future Orientation plus Performance Orientation, accounts for 20.6% of all plural rationales, or 6,732 instances. The authors argue that this pattern indicates narrow rather than substantive pluralism.

Model similarity is assessed using symmetrized Kullback–Leibler divergence,

$M = 0.123$0

with average-linkage hierarchical clustering over per-cluster preference distributions (Rahman et al., 3 Oct 2025). The resulting clusters align more closely with developer lineage than with developer geography. The paper interprets this as evidence that shared alignment strategies and corpora drive cultural behavior more strongly than national provenance.

4. CCD-Bench as a CCD detector benchmarking framework

In detector and instrumentation research, CCD-Bench refers not to a single benchmark release but to a class of benchmarking suites and test benches for charge-coupled devices. A particularly broad formulation is given through PLATOSim, the end-to-end CCD simulator created for the PLATO assessment study (Zima et al., 2010). PLATOSim simulates photometric time-series of CCD images by including realistic models of the CCD and its electronics, the telescope optics, the stellar field, ACS jitter, and natural noise sources. Its architecture produces simulated FITS time-series, extracted light curves, noise metrics versus magnitude and field density, and algorithm comparisons, and it is described as “a strong foundation for a comprehensive benchmarking suite—CCD-Bench—for space photometry missions aiming at ppm-level precision” (Zima et al., 2010).

In this usage, CCD-Bench is an end-to-end benchmarking environment for full image formation and reduction chains. PLATOSim models scene generation, white-light PSFs integrated across 400–1000 nm, flat-field and PRNU structure, jittered sub-exposure formation, zodiacal background, frame-transfer smear, readout noise, CTE degradation, and photometric extraction via aperture photometry and weighted mask photometry (Zima et al., 2010). The reported simulation results show that with 40 telescopes, PLATO can measure more than 20,000 cool dwarfs brighter than $M = 0.123$1 with precision better than 27 ppm/hour, and that algorithmic cross-over occurs around $M = 0.123$2, with aperture photometry superior at brighter magnitudes and weighted masks superior in crowded or fainter regimes.

Other detector-centered CCD-Bench implementations are laboratory rather than simulation based. For TESS, the phrase denotes the precision optical test bench developed at MIT to measure the absolute quantum efficiency of the TESS CCD detectors with high photometric stability and traceable calibration (Krishnamurthy et al., 2019). For LSST-class thick fully depleted devices, it denotes a bench-based framework combining a fast $M = 0.123$3 beam simulator with electrostatic and drift-diffusion modeling to benchmark and correct brighter-fatter, edge astrometry, tree rings, and deferred charge before first light (Bradshaw et al., 2018). For fast X-ray CCD development, Tiny-box is explicitly presented as a compact, versatile CCD characterization bench well suited to “CCD-Bench” use (Chattopadhyay et al., 2020). The TNG program likewise describes a comprehensive CCD characterization bench used to procure, assemble, and calibrate multiple detector types and controller generations (Bonano et al., 2023).

5. Instrumentation, calibration, and noise modeling across CCD benchmarking benches

Despite differences in science case, detector benchmarking benches share a common structure: controlled illumination or source generation, calibrated detector mounting, explicit treatment of readout electronics, and formal noise models. In the TESS bench, a broadband Energetiq Laser-Driven Light Source is coupled to a 20-inch integrating sphere via the CHEOPS Super Stable Source stabilization unit, with a NIST-calibrated photometer closing a feedback loop through a knife-edge attenuator (Krishnamurthy et al., 2019). The stabilized source achieves 3.57 ppm RMS in continuous mode and 4.95 ppm RMS in slow mode averaged over 60 s. A Hamamatsu S1337-1010BQ silicon photodiode, maintained at 25 °C, provides traceable radiometry; the TESS CCD is held at $M = 0.123$4 inside a vacuum dewar. Absolute QE is then obtained from

$M = 0.123$5

with $M = 0.123$6 and $M = 0.123$7 derived from calibrated photodiode power measurements (Krishnamurthy et al., 2019).

Quantum-correlation-based calibration represents a distinct metrological branch. “Absolute calibration of a CCD camera with twin beams” uses Type-II BBO spontaneous parametric down-conversion in the far field to exploit spatial pairwise quantum correlated modes (Ruo-Berchera et al., 2014). The measurement model writes

$M = 0.123$8

and extracts efficiency from the balanced-difference noise reduction factor

$M = 0.123$9

with

$\sigma = 0.109$0

Using multiple nested ROIs and full covariance treatment, the experiment reports $\sigma = 0.109$1, corresponding to a relative uncertainty of 0.3% at low photon flux (Ruo-Berchera et al., 2014).

Photon-transfer analysis provides another recurrent calibration pillar. In “The shape of the Photon Transfer Curve of CCD sensors,” variance deficits and flux-dependent covariances are linked to electrostatic redistribution via

$\sigma = 0.109$2

with the sum rule $\sigma = 0.109$3 enforcing charge conservation (Astier et al., 2019). The paper reports, for an e2v CCD250, gain $\sigma = 0.109$4, readout noise $\sigma = 0.109$5, and $\sigma = 0.109$6 averaged over 16 channels. This framework is directly proposed as a CCD-Bench characterization pipeline for unbiased gain and brighter-fatter calibration.

6. Representative implementations and benchmark outputs

The detector-oriented literature spans multiple application domains and benchmark observables.

Platform	Primary role	Representative outputs
PLATOSim (Zima et al., 2010)	End-to-end space-photometry simulation	FITS frame time-series, extracted light curves, RMS ppm/hour, crowding flags
TESS optical bench (Krishnamurthy et al., 2019)	Absolute QE measurement	QE curves for sectors A–D, gain, uncertainty budget
LSST bench framework (Bradshaw et al., 2018)	Pre-first-light CCD systematics characterization	BF slopes, centroid shifts, tree-ring maps, EPER CTI
Tiny-box (Chattopadhyay et al., 2020)	Fast X-ray CCD development	Read noise, gain, Mn K$\sigma = 0.109$7/K$\sigma = 0.109$8 spectra, dark-current trends
SPARC4 characterization (Bernardes et al., 2018)	Baseline and aging monitoring	Read noise, gain, dark current, QE spectra
TNG bench (Bonano et al., 2023)	Detector/controller procurement and commissioning	Gain from Fe-55, CTE/CTI, QE over 130–1100 nm

The SPARC4 program is especially explicit in turning routine characterization into a repeatable benchmarking protocol (Bernardes et al., 2018). Under conventional mode at $\sigma = 0.109$9 and 1 MHz readout, it reports average read noise of 6.33 electrons, average gain of 3.35 e$N = \binom{10}{2} \times 2{,}182$0/ADU, and average dark current of $N = \binom{10}{2} \times 2{,}182$1 (Bernardes et al., 2018). QE was measured from 350 nm to 1100 nm, with camera 9915 reaching 95.8% at 500 nm. The paper explicitly frames these procedures as a “systematic way for monitoring the aging of the CCDs.”

Tiny-box, by contrast, emphasizes high-speed X-ray readout development rather than static calibration (Chattopadhyay et al., 2020). At 2 MHz pixel rate and 233 K, the reported performance includes read noise of 6.95 e$N = \binom{10}{2} \times 2{,}182$2 RMS, total system noise of approximately 9 e$N = \binom{10}{2} \times 2{,}182$3 RMS at 0 ms integration, gain of approximately 2 e$N = \binom{10}{2} \times 2{,}182$4/ADU, and energy resolution of 162 eV FWHM at 5.9 keV for single-pixel grade-0 events (Chattopadhyay et al., 2020). The platform is designed for systematic sweeps of clocking profiles, digital shaping windows, bias voltages, temperature, and readout topology.

LSST-oriented benchmarking differs in emphasizing systematic image distortions relevant to weak lensing (Bradshaw et al., 2018). The bench-top $N = \binom{10}{2} \times 2{,}182$5 optical system projects realistic stars and galaxies onto single thick CCDs, and analysis with LSST Data Management pipelines quantifies brighter-fatter slopes, centroid shifts near edges, tree-ring astrometric RMS, and serial deferred charge. The benchmark outputs include adaptive second moments, flux-dependent size and shape trends, and residual ellipticity correlations relative to the LSST requirement $N = \binom{10}{2} \times 2{,}182$6 for separations of at least 5 arcmin.

7. Methodological significance and limitations

Across both meanings of CCD-Bench, the unifying methodological theme is controlled probing of failure modes that simpler tests miss. In the LLM setting, the benchmark moves beyond isolated cultural knowledge or one-axis bias diagnostics by embedding explicit clashes between value systems and requiring model selection plus rationale (Rahman et al., 3 Oct 2025). In the detector setting, CCD-Bench frameworks go beyond scalar specifications such as nominal gain or read noise by exposing interactions among optics, sampling, flat-field structure, charge transport, clocking, background, and reduction algorithms (Zima et al., 2010, Bradshaw et al., 2018, Astier et al., 2019).

The principal limitations are also domain specific. For the cultural benchmark, option-to-cluster mapping simplifies within-cluster diversity, prompts are limited to English, and rationale coding depends on explicit textual references to GLOBE dimensions (Rahman et al., 3 Oct 2025). For detector benchmarks, limitations include sub-field approximations in PLATOSim, approximate treatment of color variation and uniform background, incomplete modeling of long-term radiation evolution or cosmic rays in some settings, and device- or voltage-specific electrostatic parameters that do not transfer automatically across CCDs (Zima et al., 2010, Astier et al., 2019).

A plausible implication is that the term “CCD-Bench” now names two mature but unrelated benchmarking traditions. One is a benchmark for pluralistic decision making in LLMs, with findings of Nordic and Germanic preference, constrained rationale pluralism, negligible ordering effects, and developer-lineage clustering (Rahman et al., 3 Oct 2025). The other is a benchmark family for detector metrology and systematics qualification, encompassing end-to-end photometry simulation, absolute QE measurement, quantum-efficiency calibration with twin beams, photon-transfer modeling, X-ray readout characterization, and pre-deployment validation of precision astronomical CCDs (Zima et al., 2010, Krishnamurthy et al., 2019, Ruo-Berchera et al., 2014, Astier et al., 2019, Chattopadhyay et al., 2020, Bradshaw et al., 2018, Bernardes et al., 2018, Bonano et al., 2023).