EXCAM: Star Acquisition & Cultural Metric

• EXCAM is defined as two distinct constructs: one for precise star acquisition in Roman coronagraph operations and one as an LLM metric identifying cultural errors.
• In the Roman context, EXCAM achieves initial star positioning within ±0.054'' through direct imaging and centroiding before wavefront control.
• As an LLM evaluation tool, ExCAM generates detailed, explainable error reports by rating and identifying cultural mistakes in output text.

Searching arXiv for the provided EXCAM/ExCAM papers to ground the article in current sources. Search query: (Fathpour et al., 11 Jul 2025) EXCAM denotes two distinct technical constructs in recent arXiv literature. In Roman Coronagraph Instrument operations, EXCAM refers to the Exoplanetary System Camera star-acquisition method used at the start of a coronagraphic observation, before focal-plane-mask insertion, to place a target star within the capture range required for subsequent low-order wavefront sensing and control (Fathpour et al., 11 Jul 2025). In large-language-model evaluation, ExCAM refers to Explainable Cultural Awareness Metric, a reference-free, fine-grained metric that identifies, rates, and explains cultural errors in arbitrary instruction–output pairs, and is trained using the ExCAM40k dataset assembled from nine existing cultural benchmarks (Leiter et al., 28 May 2026).

1. Nomenclature and scope

The shared label masks a strong domain divergence. One usage belongs to observational astrophysics and spacecraft control; the other belongs to LLM evaluation and culturally aware text assessment.

Term	Domain	Core function
EXCAM	Roman Coronagraph Instrument	Initial star acquisition before FPM insertion
ExCAM	LLM evaluation	Identification, rating, and explanation of cultural errors

In the Roman context, EXCAM is embedded in a closed-loop acquisition chain that begins with spacecraft slewing and ends with hand-off to LOWFS/C, FSM control, and deformable-mirror-based dark-hole maintenance (Fathpour et al., 11 Jul 2025). In the LLM context, ExCAM is an MQM-style evaluator that outputs structured error reports and an aggregate scalar score for cultural awareness in generated text (Leiter et al., 28 May 2026).

A common source of confusion is the acronym itself. The Roman paper uses EXCAM as an instrument-acquisition term tied to the CGI science camera, whereas the LLM paper uses ExCAM as the name of an evaluation metric. No technical overlap is indicated between the two usages.

2. EXCAM in the Roman Coronagraph Instrument

Within NASA’s Nancy Grace Roman Space Telescope, the Coronagraph Instrument is a technology demonstration for direct imaging and spectroscopy of exoplanets around nearby stars at ultra-high contrast, approximately $10^{-9}$, using active wavefront control in space with two deformable mirrors (Fathpour et al., 11 Jul 2025). The observational objective imposes three coupled requirements: precise pointing onto the coronagraph focal-plane mask, active wavefront control to dig a dark hole in the stellar PSF, and LOWFS/C stabilization of tip/tilt and focus drifts.

EXCAM is used only at the very start of each coronagraphic observation, when the focal-plane mask is not yet inserted and the full science-camera field of view is unobscured at $19.5'' \times 19.5''$ (Fathpour et al., 11 Jul 2025). The observatory Attitude Control System first slews to the commanded reference star and, per the CGI–ACS interface, must place that star within a $4''$ radius $(3\sigma)$ of EXCAM’s center pixel. EXCAM then performs direct imaging and centroiding to refine the line of sight until the star lies within the fine-point capture range required for hand-off.

This stage is operationally significant because it furnishes the initial condition for the downstream coronagraphic control stack. EXCAM acquisition alone places the star to within $\pm 0.054''$ of the nominal LoS axis, ensuring that the target lies within the linear capture range of the LOWFS Tip/Tilt $(Z2/Z3)$ sensor (Fathpour et al., 11 Jul 2025). The paper is explicit that EXCAM itself does not command the deformable mirrors; its function is acquisition and alignment, not dark-hole control.

3. Acquisition workflow and control interfaces

The EXCAM star-acquisition sequence is implemented in CGI flight software and proceeds through command setup, initial spacecraft pointing, frame acquisition and preprocessing, star identification, centroid computation, capture-range testing, ACS offload, repointing, iteration, and hand-off to LOWFS/C (Fathpour et al., 11 Jul 2025). Command setup includes configuring detector gain and exposure time according to stellar visual magnitude $(V \le 5)$, updating and loading the appropriate master-dark frame, and selecting both the number of exposures $(Nframes)$ and a photometric threshold for star detection.

Image processing applies cosmic-ray cleaning and dark subtraction to each frame, followed by frame co-addition or averaging to improve SNR. Candidate stars are identified as local maxima whose peak intensity satisfies $I_{\text{peak}} > I_{\text{thresh}}$, where $I_{\text{thresh}}$ is set such that any confusing star $19.5'' \times 19.5''$0 is rejected. The centroid of a candidate star is then computed over a local region of interest by intensity-weighted first moments: $19.5'' \times 19.5''$1 The single-frame signal-to-noise ratio is approximated as

$19.5'' \times 19.5''$2

and $19.5'' \times 19.5''$3 are chosen so that $19.5'' \times 19.5''$4, with $19.5'' \times 19.5''$5, for reliable centroiding.

The fine-point capture range is $19.5'' \times 19.5''$6 pixels, approximately $19.5'' \times 19.5''$7 radius. If the centroid lies outside this capture circle, EXCAM computes pointing error offsets, converts pixel offsets $19.5'' \times 19.5''$8 into spacecraft LoS-frame rotations $19.5'' \times 19.5''$9 in radians via the plate scale $4''$0 pixels and the CGI LoS-frame definition, and sends a “single-delta-HV” offload packet at $4''$1 to ACS (Fathpour et al., 11 Jul 2025). ACS uses these corrections, slews the telescope, and flags “AT Offset” to FALSE until pointing settles. On ACS “hold attitude” flag TRUE, the frame-processing and offload cycle is repeated until the centroid lies within capture range.

After acquisition success, the focal-plane mask is inserted immediately and the system switches to LOCAM. At that stage the LOWFS/C FPGA closes a high-speed $4''$2 FSM inner loop for tip/tilt, performs periodic piezo offloads to ACS to prevent FSM saturation, estimates Zernike modes beyond tip/tilt from the occulted pupil image, and issues deformable-mirror commands to correct quasi-static wavefront errors and maintain the dark hole (Fathpour et al., 11 Jul 2025). This clarifies the division of labor: EXCAM establishes the initial line-of-sight geometry; LOWFS/C and the DMs sustain coronagraphic performance.

4. Thermal-vacuum demonstration and operational trade-offs

The full-system thermal-vacuum campaign at JPL exercised EXCAM acquisition in HLC NFOV mode using the Coronagraph Verification Stimulus to emulate star input and ACS response (Fathpour et al., 11 Jul 2025). The reported performance verifies the relevant acquisition requirements: ACS pre-pointing error was at most $4''$3 radius $4''$4, the EXCAM capture-range requirement of at most $4''$5 radius was verified, StarID met the $4''$6 requirement, and FindCentroid met the $4''$7 requirement. End-to-end acquisition time was $4''$8 minutes for bright $4''$9 stars placed at initial offsets of $(3\sigma)$0 and $(3\sigma)$1, and final pointing was at most $(3\sigma)$2 from the EXCAM boresight.

The TVAC error budget partitions residual acquisition uncertainty across multiple subsystems: ACS boresight and catalog calibration contribute approximately $(3\sigma)$3 RMS, observatory jitter contributes approximately $(3\sigma)$4 RMS, and centroid error contributes approximately $(3\sigma)$5, with SNR dependence (Fathpour et al., 11 Jul 2025). All test cases met requirements, including bright $(3\sigma)$6 stars, dim $(3\sigma)$7 stars, and operation under simulated ACS disturbances such as reaction-wheel zero-crossing and HGA moves.

The paper contrasts EXCAM with the Raster Scan method. EXCAM offers direct imaging of the star, simple centroiding, the full $(3\sigma)$8 FoV, and fast initialization of LoS pointing in $(3\sigma)$9 minutes, while maintaining lower complexity because no FSM raster pattern is required (Fathpour et al., 11 Jul 2025). Its limitations are equally explicit: it is only viable before FPM insertion and requires relatively bright stars $\pm 0.054''$0 to achieve adequate SNR in a few-second exposures. Raster Scan, by contrast, operates with the FPM in place and supports mid-sequence target reacquisition, including fainter stars behind the bowtie mask, but it requires more complex raster-trajectory design, careful FSM strain-gauge calibration, dark-frame handling, and longer routines, up to approximately $\pm 0.054''$1 for the raster plus loop closing.

The operational lessons are primarily software and interface lessons. Early hardware-in-the-loop testing, adjustable flight-software parameters such as photometric threshold and capture-range margin, rapid master-dark and gain updates, comprehensive telemetry logging, minimal perturbation before dark-hole preparation, and robust state-machine handling for ACS disturbance flags are all identified as practical recommendations (Fathpour et al., 11 Jul 2025). This suggests that EXCAM’s importance lies not only in centroiding accuracy but also in system-level fault tolerance and commissioning maturity.

5. ExCAM as an explainable cultural-awareness metric

ExCAM, the Explainable Cultural Awareness Metric, is defined as the first reference-free, fine-grained metric designed specifically to identify, rate, and explain cultural errors in arbitrary instruction–output pairs produced by LLMs (Leiter et al., 28 May 2026). It formalizes cultural awareness as the absence of culturally problematic content, including factual mistakes, misrepresentations, stereotypes, and other errors that would mislead or offend members of a target culture.

Given an instruction $\pm 0.054''$2 and an LLM-generated output $\pm 0.054''$3, ExCAM returns an MQM-style error report $\pm 0.054''$4 and a scalar quality score $\pm 0.054''$5. Each entry in $\pm 0.054''$6 contains an error type, a span, a severity label with minor $\pm 0.054''$7 and major $\pm 0.054''$8, and a natural-language explanation. The scalar score is defined by

$\pm 0.054''$9

and

$(Z2/Z3)$0

where $(Z2/Z3)$1 is the length-normalized probability assigned by the LLM to the entire error report (Leiter et al., 28 May 2026). A perfect, error-free output therefore attains $(Z2/Z3)$2 when the model is maximally confident, while each major error deducts $(Z2/Z3)$3 points and each minor error deducts $(Z2/Z3)$4 point.

Methodologically, ExCAM is not a single monolithic model but a suite of LLM evaluators based on Gemma 3-27B and Phi 3-medium-4k adapted via LoRA fine-tuning. Input is wrapped in a prompt of the form Instruction: {i}\nText: {o}\nReturn an error report in JSON., and robustness is promoted with 50 automatically generated paraphrases of these templates (Leiter et al., 28 May 2026). The JSON schema contains error_type, span, severity, and explanation, making the metric inherently interpretable.

Negative supervision is produced with synthetic error generation. “Hard errors” are drawn from existing QA benchmarks by selecting incorrect answer options and assigning severity $(Z2/Z3)$5 with templated explanations. “Soft errors” are generated by prompting Qwen3.5-122B in “thinking” mode to introduce either a minor or major cultural error into a correct instruction–output pair, together with an error type, explanation, severity, and modified sample, while discarding outputs no longer judged culture-related (Leiter et al., 28 May 2026). A key misconception corrected by the paper is that cultural evaluation must rely on reference answers or manual annotation at inference time; ExCAM is explicitly reference-free at evaluation time.

6. ExCAM40k, empirical performance, and explainability

ExCAM40k consolidates nine human-verified cultural benchmarks into an instruction–output format and augments them with synthetic errors (Leiter et al., 28 May 2026). The sources are grouped as QA with fixed options—BLEnD, CulturalBench, INCLUDE; free-form QA—CaLMQA, NativQA; free-text generation—Mango; and impersonation tasks—Normad, GlobalOpinionQA, EPIC. After capping each source at $(Z2/Z3)$6 samples, the corpus contains roughly $(Z2/Z3)$7 error-free examples, $(Z2/Z3)$8 hard-error samples, and $(Z2/Z3)$9 soft-error samples, yielding approximately $(V \le 5)$0 instances split into training, development, and test partitions.

The annotation pipeline assigns empty error reports to original human-verified pairs, introduces hard or soft errors as negative samples, records error spans by token-sequence diffing with NLTK and difflib, and treats the synthetic reports as ground truth. A human validation of 100 randomly sampled soft errors found that $(V \le 5)$1 were truly culture-related, $(V \le 5)$2 of generated explanations were valid, and severity labels matched in $(V \le 5)$3 of minor cases and $(V \le 5)$4 of major cases (Leiter et al., 28 May 2026). These figures delimit both the promise and the residual subjectivity of synthetic cultural-error generation.

Evaluation uses scaled accuracy, Kendall’s $(V \le 5)$5, and tie-calibrated accuracy, with LLM-based baselines including Qwen3-14B, DeepSeek-R1-Distill-Qwen-14B, Gemma 3-27B, Mistral-24B, Phi 3-medium-4k, Phi 4-mini, and GPT-5, and with counting prompts outperforming binary prompts (Leiter et al., 28 May 2026). On the in-domain test set, the Gemma-based ExCAM variant achieves 0.591 scaled accuracy, 0.569 Kendall’s $(V \le 5)$6, and 0.680 tie-calibrated accuracy, compared with 0.351, 0.349, and 0.495 for Phi 4$(V \le 5)$7, the strongest unfine-tuned baseline listed. The paper further states that the Gemma-based variant attains a raw error-detection accuracy of approximately $(V \le 5)$8 and that its Kendall correlation is significantly higher $(V \le 5)$9 than that of any baseline.

Generalization is evaluated through leave-one-out LoRA training, omitting each source dataset in turn and testing on its held-out split. On six of nine benchmarks—BLEnD, CulturalBench, INCLUDE, Mango, NativQA, and Normad—ExCAM maintains significantly better scaled accuracy than any unfine-tuned baseline (Leiter et al., 28 May 2026). On a balanced 900-sample subset, ExCAM$(Nframes)$0 significantly outperforms GPT-5 on six out of nine benchmarks. The paper interprets this as evidence that a dedicated, fine-tuned metric can exceed a general-purpose LLM in nuanced cultural-error detection.

Explainability is central rather than incidental. Because ExCAM returns an error report rather than a single scalar, it specifies what the error is, where it appears, how severe it is, and why it is wrong. In a spot-check of $(Nframes)$1 of hard-error cases, ExCAM flagged the absence of the correct answer, and in $(Nframes)$2 of its explanations it literally reproduced the one-to-one correct answer as the remedy (Leiter et al., 28 May 2026). In soft-error cases, human raters assigned its explanations an average Likert score of $(Nframes)$3. Representative examples include free-text overgeneralization such as “No Germans like schnitzel,” labeled as an overgeneralization with severity $(Nframes)$4, and impersonation mismatch such as a Malaysian sleep-response judged atypical and labeled with severity $(Nframes)$5.

The future directions identified by the paper are severity calibration with richer human labels, real-world error corpora beyond synthetic perturbations, ensembling and debiasing across LLM families to mitigate self-bias, and extension to broader modalities such as image or multimodal outputs (Leiter et al., 28 May 2026). A plausible implication is that ExCAM’s long-term significance depends not only on benchmark performance but also on whether its explanation schema becomes stable enough to support iterative model debugging, alignment audits, and culturally conditioned error analysis across domains.