---
title: 'CCD-Bench: Dual Benchmarking in LLMs & CCD Detectors'
url: https://www.emergentmind.com/topics/ccd-bench
type: topic
---

# CCD-Bench: Dual Benchmarking in LLMs & CCD Detectors

Searching arXiv for "CCD-Bench" and closely related CCD benchmarking work.
CCD-Bench is an ambiguous label used in two distinct research lineages on arXiv. In one lineage, “CCD-Bench” denotes the “Culture-Conflict Decision Benchmark,” a benchmark for evaluating large language model decision-making under explicit cross-cultural value conflict [2510.03553]. In another lineage, the same label is used descriptively for benchmarking infrastructures for charge-coupled device characterization, calibration, and systematics studies, spanning space photometry simulators, optical quantum-efficiency benches, X-ray detector test stands, photon-transfer analysis, and LSST-style laboratory systematics campaigns [1004.0738], [1911.09270], [1808.00534], [1405.1526], [2012.07161], [1905.08677], [1806.02191], [2302.11226]. The term therefore does not designate a single canonical artifact across the literature; rather, it refers either to a cultural-conflict benchmark for LLMs or to a family of CCD benchmarking frameworks for detector performance and calibration.

## 1. Terminological scope and disambiguation

The most explicit contemporary use of the name appears in “CCD-Bench: Probing Cultural Conflict in Large Language Model Decision-Making,” where CCD-Bench is expanded as “Culture-Conflict Decision Benchmark” and is designed to evaluate how LLMs choose among culturally grounded, mutually legitimate response options under overt value conflict [2510.03553]. The benchmark targets decision-making rather than cultural recall, demographic value prediction, or single-axis bias scoring.

A separate usage emerges in instrumentation and detector-characterization literature, where “CCD-Bench” functions as a descriptive name for a benchmarking suite or test bench centered on charge-coupled devices. In this sense, the term is used for precision optical benches, X-ray detector stands, laboratory campaigns for systematics characterization, and end-to-end simulation environments that can be translated into benchmark scenarios and metrics [1911.09270], [2012.07161], [1808.00534], [1004.0738]. This suggests that “CCD-Bench” is polysemous across fields, with one branch rooted in LLM evaluation and another in CCD detector metrology and calibration.

## 2. CCD-Bench as the Culture-Conflict Decision Benchmark

In the LLM literature, CCD-Bench comprises 2,182 open-ended dilemmas spanning seven domains: Arts, Education, Family, Wellness, Work, Scientific, and Lifestyle [2510.03553]. Each dilemma is paired with exactly ten anonymized response options mapped to the ten GLOBE cultural clusters representing 62 societies: Anglo, Latin Europe, Nordic Europe, Germanic Europe, Eastern Europe, Latin America, Sub-Saharan Africa, Middle East and North Africa, Southern Asia, and Confucian Asia. The benchmark is constructed to expose models to “value-plural dilemmas where several culturally grounded solutions are equally defensible,” then require the model to choose and justify one anonymized option.

Dataset construction follows a two-step pipeline in which a state-of-the-art reasoning model generates, for each cluster, a one-sentence answer that does not mention the culture and a one-sentence rationale that references cultural values. Similarity pruning uses pairwise MinHash Jaccard similarity over 3-gram shingles within each 10-tuple, with regeneration triggered when similarity exceeds $\tau = 0.70$ [2510.03553]. Across the final corpus, the pairwise MinHash Jaccard similarity among alternatives has mean $M = 0.123$ and standard deviation $\sigma = 0.109$ over $N = \binom{10}{2} \times 2{,}182$ comparisons. Human verification confirms that each option is reasonable within the target cluster, that no option names the cluster explicitly, and that options are semantically distinct.

To mitigate ordering effects, CCD-Bench uses a Stratified Latin Square design. For each domain separately, a $10 \times 10$ Latin square determines display order, so that each cluster appears exactly once in every ordinal slot and precedes or follows every other cluster exactly once across the corpus. The reported Latin-square balance score is approximately 99.8%, and ordering effects are negligible according to chi-squared tests and Cramér’s $V$, with all clusters yielding $V < 0.10$ [2510.03553].

## 3. Evaluation protocol and findings in the LLM benchmark

The benchmark evaluates 17 leading non-reasoning LLMs available via OpenRouter, with “non-reasoning” defined as evaluation without explicit chain-of-thought prompting, tool use, or auxiliary retrieval [2510.03553]. For each dilemma, models see the ten anonymized options in Stratified Latin Square order and are asked to select one option and provide a justification. Aggregation proceeds by counting selections per cluster globally and per domain, using raw tallies without smoothing or normalization. The paper reports total valid selections of 36,977 and 37,092 rationales out of the theoretical maximum of 37,094.

The principal empirical result is a marked skew in cultural preference distributions. Across the 17 models, Nordic Europe is disproportionately preferred, with mean 20.2% of selections, while Germanic Europe averages 12.4%, Sub-Saharan Africa 11.5%, and Anglo 11.3% [2510.03553]. Eastern Europe at 5.6% and Middle East and North Africa at 5.8% are underrepresented, and no model exceeds 8.2% for either cluster. The paper further reports that Latin Europe shows the smallest coefficient of variation, $CV = 0.09$, while Anglo exhibits the largest, $CV = 0.23$.

Rationale analysis codes explicit mentions of GLOBE dimensions in model-generated justifications. Across $N = 37{,}092$ rationales, 87.9% reference two or more dimensions, which the paper interprets as formal pluralism [2510.03553]. However, dimension usage is highly concentrated: Performance Orientation is 16.2%, Future Orientation 15.1%, Institutional Collectivism 14.9%, In-Group Collectivism 13.5%, Humane Orientation 13.0%, Power Distance 12.5%, Uncertainty Avoidance 9.5%, Gender Egalitarianism 2.7%, and Assertiveness 2.6%. A single pairing, Future Orientation plus Performance Orientation, accounts for 20.6% of all plural rationales, or 6,732 instances. The authors argue that this pattern indicates narrow rather than substantive pluralism.

Model similarity is assessed using symmetrized Kullback–Leibler divergence,
$$
D_{\mathrm{SKL}}(P,Q)=\frac{1}{2}\left[KL(P\|Q)+KL(Q\|P)\right],
$$
with average-linkage hierarchical clustering over per-cluster preference distributions [2510.03553]. The resulting clusters align more closely with developer lineage than with developer geography. The paper interprets this as evidence that shared alignment strategies and corpora drive cultural behavior more strongly than national provenance.

## 4. CCD-Bench as a CCD detector benchmarking framework

In detector and instrumentation research, CCD-Bench refers not to a single benchmark release but to a class of benchmarking suites and test benches for charge-coupled devices. A particularly broad formulation is given through PLATOSim, the end-to-end CCD simulator created for the PLATO assessment study [1004.0738]. PLATOSim simulates photometric time-series of CCD images by including realistic models of the CCD and its electronics, the telescope optics, the stellar field, ACS jitter, and natural noise sources. Its architecture produces simulated FITS time-series, extracted light curves, noise metrics versus magnitude and field density, and algorithm comparisons, and it is described as “a strong foundation for a comprehensive benchmarking suite—CCD-Bench—for space photometry missions aiming at ppm-level precision” [1004.0738].

In this usage, CCD-Bench is an end-to-end benchmarking environment for full image formation and reduction chains. PLATOSim models scene generation, white-light PSFs integrated across 400–1000 nm, flat-field and PRNU structure, jittered sub-exposure formation, zodiacal background, frame-transfer smear, readout noise, CTE degradation, and photometric extraction via aperture photometry and weighted mask photometry [1004.0738]. The reported simulation results show that with 40 telescopes, PLATO can measure more than 20,000 cool dwarfs brighter than $m_V = 11$ with precision better than 27 ppm/hour, and that algorithmic cross-over occurs around $m_V \approx 12$, with aperture photometry superior at brighter magnitudes and weighted masks superior in crowded or fainter regimes.

Other detector-centered CCD-Bench implementations are laboratory rather than simulation based. For TESS, the phrase denotes the precision optical test bench developed at MIT to measure the absolute quantum efficiency of the TESS CCD detectors with high photometric stability and traceable calibration [1911.09270]. For LSST-class thick fully depleted devices, it denotes a bench-based framework combining a fast $f/1.2$ beam simulator with electrostatic and drift-diffusion modeling to benchmark and correct brighter-fatter, edge astrometry, tree rings, and deferred charge before first light [1808.00534]. For fast X-ray CCD development, Tiny-box is explicitly presented as a compact, versatile CCD characterization bench well suited to “CCD-Bench” use [2012.07161]. The TNG program likewise describes a comprehensive CCD characterization bench used to procure, assemble, and calibrate multiple detector types and controller generations [2302.11226].

## 5. Instrumentation, calibration, and noise modeling across CCD benchmarking benches

Despite differences in science case, detector benchmarking benches share a common structure: controlled illumination or source generation, calibrated detector mounting, explicit treatment of readout electronics, and formal noise models. In the TESS bench, a broadband Energetiq Laser-Driven Light Source is coupled to a 20-inch integrating sphere via the CHEOPS Super Stable Source stabilization unit, with a NIST-calibrated photometer closing a feedback loop through a knife-edge attenuator [1911.09270]. The stabilized source achieves 3.57 ppm RMS in continuous mode and 4.95 ppm RMS in slow mode averaged over 60 s. A Hamamatsu S1337-1010BQ silicon photodiode, maintained at 25 °C, provides traceable radiometry; the TESS CCD is held at $-70\,^\circ\mathrm{C}$ inside a vacuum dewar. Absolute QE is then obtained from
$$
\mathrm{QE}(\lambda)=\frac{N_e(\lambda)}{N_\gamma(\lambda)},
$$
with $N_e = \mathrm{ADU}\times G$ and $N_\gamma$ derived from calibrated photodiode power measurements [1911.09270].

Quantum-correlation-based calibration represents a distinct metrological branch. “Absolute calibration of a CCD camera with twin beams” uses Type-II BBO spontaneous parametric down-conversion in the far field to exploit spatial pairwise quantum correlated modes [1405.1526]. The measurement model writes
$$
m_s = G_s[\eta_s T_s n_s] + b_s + u_s,\qquad
m_i = G_i[\eta_i T_i n_i] + b_i + u_i,
$$
and extracts efficiency from the balanced-difference noise reduction factor
$$
\sigma_\alpha \equiv \frac{\langle[\delta(N_i-\alpha N_s)]^2\rangle}{\langle N_i+\alpha N_s\rangle},
$$
with
$$
\sigma_\alpha = \frac{1+\alpha}{2}-\eta_i A.
$$
Using multiple nested ROIs and full covariance treatment, the experiment reports $\langle \eta_i\rangle = 0.725(0.002)$, corresponding to a relative uncertainty of 0.3% at low photon flux [1405.1526].

Photon-transfer analysis provides another recurrent calibration pillar. In “The shape of the Photon Transfer Curve of CCD sensors,” variance deficits and flux-dependent covariances are linked to electrostatic redistribution via
$$
C_{ij}(\mu)=\frac{\mu}{g}\left[\delta_{i0}\delta_{j0}+a_{ij}\mu g+\frac{2}{3}[\mathbf{a}\otimes\mathbf{a}]_{ij}(\mu g)^2+\cdots\right]+\frac{n_{ij}}{g^2},
$$
with the sum rule $\sum_{kl} a_{kl}=0$ enforcing charge conservation [1905.08677]. The paper reports, for an e2v CCD250, gain $0.713\ \mathrm{el/ADU}$, readout noise $4.54\ \mathrm{el}$, and $a_{00}=-2.377\times 10^{-6}\ \mathrm{el}^{-1}$ averaged over 16 channels. This framework is directly proposed as a CCD-Bench characterization pipeline for unbiased gain and brighter-fatter calibration.

## 6. Representative implementations and benchmark outputs

The detector-oriented literature spans multiple application domains and benchmark observables.

| Platform | Primary role | Representative outputs |
|---|---|---|
| PLATOSim [1004.0738] | End-to-end space-photometry simulation | FITS frame time-series, extracted light curves, RMS ppm/hour, crowding flags |
| TESS optical bench [1911.09270] | Absolute QE measurement | QE curves for sectors A–D, gain, uncertainty budget |
| LSST bench framework [1808.00534] | Pre-first-light CCD systematics characterization | BF slopes, centroid shifts, tree-ring maps, EPER CTI |
| Tiny-box [2012.07161] | Fast X-ray CCD development | Read noise, gain, Mn K$\alpha$/K$\beta$ spectra, dark-current trends |
| SPARC4 characterization [1806.02191] | Baseline and aging monitoring | Read noise, gain, dark current, QE spectra |
| TNG bench [2302.11226] | Detector/controller procurement and commissioning | Gain from Fe-55, CTE/CTI, QE over 130–1100 nm |

The SPARC4 program is especially explicit in turning routine characterization into a repeatable benchmarking protocol [1806.02191]. Under conventional mode at $-70\,^\circ\mathrm{C}$ and 1 MHz readout, it reports average read noise of 6.33 electrons, average gain of 3.35 e$^-$/ADU, and average dark current of $0.00014\ \mathrm{e^-\,pix^{-1}\,s^{-1}}$ [1806.02191]. QE was measured from 350 nm to 1100 nm, with camera 9915 reaching 95.8% at 500 nm. The paper explicitly frames these procedures as a “systematic way for monitoring the aging of the CCDs.”

Tiny-box, by contrast, emphasizes high-speed X-ray readout development rather than static calibration [2012.07161]. At 2 MHz pixel rate and 233 K, the reported performance includes read noise of 6.95 e$^-$ RMS, total system noise of approximately 9 e$^-$ RMS at 0 ms integration, gain of approximately 2 e$^-$/ADU, and energy resolution of 162 eV FWHM at 5.9 keV for single-pixel grade-0 events [2012.07161]. The platform is designed for systematic sweeps of clocking profiles, digital shaping windows, bias voltages, temperature, and readout topology.

LSST-oriented benchmarking differs in emphasizing systematic image distortions relevant to weak lensing [1808.00534]. The bench-top $f/1.2$ optical system projects realistic stars and galaxies onto single thick CCDs, and analysis with LSST Data Management pipelines quantifies brighter-fatter slopes, centroid shifts near edges, tree-ring astrometric RMS, and serial deferred charge. The benchmark outputs include adaptive second moments, flux-dependent size and shape trends, and residual ellipticity correlations relative to the LSST requirement $E=\langle \delta e\cdot \delta e\rangle \le 2\times 10^{-7}$ for separations of at least 5 arcmin.

## 7. Methodological significance and limitations

Across both meanings of CCD-Bench, the unifying methodological theme is controlled probing of failure modes that simpler tests miss. In the LLM setting, the benchmark moves beyond isolated cultural knowledge or one-axis bias diagnostics by embedding explicit clashes between value systems and requiring model selection plus rationale [2510.03553]. In the detector setting, CCD-Bench frameworks go beyond scalar specifications such as nominal gain or read noise by exposing interactions among optics, sampling, flat-field structure, charge transport, clocking, background, and reduction algorithms [1004.0738], [1808.00534], [1905.08677].

The principal limitations are also domain specific. For the cultural benchmark, option-to-cluster mapping simplifies within-cluster diversity, prompts are limited to English, and rationale coding depends on explicit textual references to GLOBE dimensions [2510.03553]. For detector benchmarks, limitations include sub-field approximations in PLATOSim, approximate treatment of color variation and uniform background, incomplete modeling of long-term radiation evolution or cosmic rays in some settings, and device- or voltage-specific electrostatic parameters that do not transfer automatically across CCDs [1004.0738], [1905.08677].

A plausible implication is that the term “CCD-Bench” now names two mature but unrelated benchmarking traditions. One is a benchmark for pluralistic decision making in LLMs, with findings of Nordic and Germanic preference, constrained rationale pluralism, negligible ordering effects, and developer-lineage clustering [2510.03553]. The other is a benchmark family for detector metrology and systematics qualification, encompassing end-to-end photometry simulation, absolute QE measurement, quantum-efficiency calibration with twin beams, photon-transfer modeling, X-ray readout characterization, and pre-deployment validation of precision astronomical CCDs [1004.0738], [1911.09270], [1405.1526], [1905.08677], [2012.07161], [1808.00534], [1806.02191], [2302.11226].

Source: https://www.emergentmind.com/topics/ccd-bench