Selective Reflection in Optics & Materials

• Selective reflection is the resonant modification of light at material interfaces, revealing sub-Doppler structures and surface-dependent optical responses.
• It encompasses diverse systems—from atomic vapors and nanocells to cholesteric liquid crystals and superconductors—using interference, Fabry–Perot, and Bragg reflection mechanisms.
• Advanced methods, including frequency modulation and pump–probe schemes, yield narrow spectral features and tunable resonance profiles vital for precision spectroscopy and device engineering.

Selective reflection is a term used in several specialized literatures. In atomic and interface optics, it denotes the resonant modification of light reflected from a dielectric–vapor or dielectric–resonant-medium boundary, with the reflected signal governed by the near-surface optical response of the adjacent medium and often exhibiting sub-Doppler structure (Khachatryan, 2018, Martins et al., 2013). In cholesteric liquid crystals, it denotes Bragg reflection produced by the helical modulation of optical anisotropy, with the reflected wavelength set primarily by the pitch (Xiang et al., 2015, Lebovka et al., 2013). The term also appears in other technical contexts, including spin-selective equal-spin Andreev reflection, circular-polarization-selective perfect reflection, frequency-selective reflection by reconfigurable intelligent surfaces, and a teacher–student data-recycling framework called Selective Reflection-Tuning (Li et al., 2017, Ahn et al., 3 Feb 2025, Souza et al., 21 Aug 2025, Li et al., 2024).

1. Atomic selective reflection at dielectric interfaces

In the atomic-optics sense, selective reflection occurs when light is incident on the interface between a transparent dielectric and a resonant vapor. Near an atomic resonance, the vapor refractive index becomes frequency dependent, and this produces a small frequency-dependent change in the reflected light intensity (Martins et al., 2013). A compact formulation writes the interface reflection amplitude as

$r=\frac{n-1}{n+1}-\frac{n}{(n+1)^2}\,\bar{\chi},$

with $n$ the refractive index of the dielectric window and $\bar{\chi}$ the effective vapor susceptibility seen in reflection (Dutta et al., 8 Jul 2025). To first order, the reflected intensity is

$$R=|r|^2=\left(\frac{n-1}{n+1}\right)^2-\frac{2n(n-1)}{(n+1)^3}\,\Re[\bar{\chi}],$$

so the resonant part of the signal is a perturbation of the ordinary Fresnel reflection (Dutta et al., 8 Jul 2025).

The effective susceptibility is nonlocal. One formulation is

$$\chi=-\frac{2ikN\mu}{\epsilon_0E}\int_0^{\infty}dz\int dv_z\,W(v_z)\sigma_{ge}(z,v_z)e^{2ikz},$$

where the factor $e^{2ikz}$ weights the contribution of dipoles according to the phase of their backward reradiated field (Martins et al., 2013). Because contributions from increasing depth dephase, selective reflection is dominated by atoms in the immediate vicinity of the surface. Another equivalent expression uses

$$\bar{\chi}=-\frac{2ik}{\varepsilon_0 E_T}\int_0^\infty dz\,p(z)e^{2ikz},$$

again making the near-interface sensitivity explicit (Dutta et al., 8 Jul 2025).

This surface selectivity is also what differentiates selective reflection from ordinary transmission or absorption spectroscopy. Transmission integrates over the full vapor column, whereas selective reflection probes a region of order $\lambda/2\pi$ adjacent to the interface (Sargsyan et al., 2024). In resonant-vapor spectroscopy this produces sub-Doppler structure, makes the method sensitive to atom–surface interactions, and permits measurements in geometries where bulk transmission is weak or strongly absorbing (Martins et al., 2013, Chan et al., 2024).

2. Spectral formation, interference, and asymmetric line shapes

The line shape of selective reflection is controlled by a combination of transient atomic polarization, interface optics, and, in finite cells, Fabry–Perot interference. A self-consistent theory for a dilute vapor cell treats the system as a glass$|$vapor$|$glass Fabry–Perot interferometer with wall-induced spatial dispersion (Khachatryan, 2018). In that treatment, diffusive wall collisions reset the polarization, leading to a nonlocal kernel

$n$0

and the polarization at a point depends on atoms arriving from each wall (Khachatryan, 2018). The full reflected amplitude can then be written in a Fabry–Perot-like form,

$n$1

with effective propagation constants $n$2, interface coefficients $n$3, and cavity phase $n$4 (Khachatryan, 2018). In the single-interface limit, the reflection reduces to

$n$5

so one can define an effective complex refractive index for single selective reflection (Khachatryan, 2018).

A distinct asymmetry mechanism appears in resonant media consisting of two-level atoms embedded in a dielectric host. In the thin-layer limit, with normal incidence and negligible propagation inside the layer, the reflection amplitude is the Fresnel coefficient

$n$6

where the dielectric function contains both a nonresonant host contribution and a resonant atomic contribution (Novitsky, 2011). In that model the asymmetric selective-reflection line is interpreted as a Fano-like resonance: a narrow resonant pathway from the embedded atoms interferes with a smooth nonresonant background pathway from the dielectric mismatch (Novitsky, 2011). The asymmetry parameter analogue is the refractive-index difference

$n$7

When $n$8, the resonance is symmetric; when $n$9, it becomes asymmetric, and the sign of $\bar{\chi}$0 determines the direction of skewness (Novitsky, 2011). The same model also shows that local-field corrections can shift and distort the resonance and can lead to intrinsic optical bistability through the cubic stationary inversion equation (Novitsky, 2011).

These two strands of theory address complementary regimes. The Fabry–Perot treatment emphasizes nonlocal transient polarization and multiple reflections in dilute vapor cells, whereas the Fano-like treatment isolates interference between resonant and background reflection channels in a thin composite resonant medium (Khachatryan, 2018, Novitsky, 2011).

3. Nanocells, derivative techniques, and atom–surface spectroscopy

Nanocells make selective reflection especially powerful because the vapor thickness is comparable to or much smaller than the optical wavelength. In a cesium nanocell of thickness $\bar{\chi}$1 nm on the $\bar{\chi}$2 line, the real-time derivative of the selective-reflection signal, the “D-peak,” yields linewidths of about $\bar{\chi}$3 MHz, roughly ten times narrower than the Doppler width at $\bar{\chi}$4C and more than twice narrower than the corresponding absorption signal (Sargsyan et al., 2016). In that geometry the authors tracked 28 Zeeman transitions over $\bar{\chi}$5 kG and observed the collapse to 8 transitions in the hyperfine Paschen–Back regime (Sargsyan et al., 2016).

In ultrathin rubidium cells, selective reflection becomes a sensitive probe of atom–surface forces. For a cell thickness $\bar{\chi}$6 nm on the Rb $\bar{\chi}$7 line, the derivative of selective reflection (DSR) exhibits strong red shifts and broadening caused by van der Waals interaction with the two nearby dielectric walls (Sargsyan et al., 2017). The paper uses

$\bar{\chi}$8

with the center estimate

$\bar{\chi}$9

At $$R=|r|^2=\left(\frac{n-1}{n+1}\right)^2-\frac{2n(n-1)}{(n+1)^3}\,\Re[\bar{\chi}],$$0 nm the reported zero-field red shift is $$R=|r|^2=\left(\frac{n-1}{n+1}\right)^2-\frac{2n(n-1)}{(n+1)^3}\,\Re[\bar{\chi}],$$1 MHz and the DSR linewidth is about $$R=|r|^2=\left(\frac{n-1}{n+1}\right)^2-\frac{2n(n-1)}{(n+1)^3}\,\Re[\bar{\chi}],$$2 MHz (Sargsyan et al., 2017). Even so, the method still resolves the hyperfine Paschen–Back structure for $$R=|r|^2=\left(\frac{n-1}{n+1}\right)^2-\frac{2n(n-1)}{(n+1)^3}\,\Re[\bar{\chi}],$$3 kG, with four components for $$R=|r|^2=\left(\frac{n-1}{n+1}\right)^2-\frac{2n(n-1)}{(n+1)^3}\,\Re[\bar{\chi}],$$4Rb and six for $$R=|r|^2=\left(\frac{n-1}{n+1}\right)^2-\frac{2n(n-1)}{(n+1)^3}\,\Re[\bar{\chi}],$$5Rb (Sargsyan et al., 2017).

Potassium nanocells show the same methodological pattern. On the K $$R=|r|^2=\left(\frac{n-1}{n+1}\right)^2-\frac{2n(n-1)}{(n+1)^3}\,\Re[\bar{\chi}],$$6 line, the derivative SR signal reaches a linewidth of about $$R=|r|^2=\left(\frac{n-1}{n+1}\right)^2-\frac{2n(n-1)}{(n+1)^3}\,\Re[\bar{\chi}],$$7 MHz for $$R=|r|^2=\left(\frac{n-1}{n+1}\right)^2-\frac{2n(n-1)}{(n+1)^3}\,\Re[\bar{\chi}],$$8 nm, which is 18 times narrower than the Doppler linewidth of about $$R=|r|^2=\left(\frac{n-1}{n+1}\right)^2-\frac{2n(n-1)}{(n+1)^3}\,\Re[\bar{\chi}],$$9 MHz (Sargsyan et al., 2019). The same work reports a $$\chi=-\frac{2ikN\mu}{\epsilon_0E}\int_0^{\infty}dz\int dv_z\,W(v_z)\sigma_{ge}(z,v_z)e^{2ikz},$$0-periodic sign oscillation of the derivative signal with thickness and extracts the first potassium van der Waals coefficient near sapphire,

$$\chi=-\frac{2ikN\mu}{\epsilon_0E}\int_0^{\infty}dz\int dv_z\,W(v_z)\sigma_{ge}(z,v_z)e^{2ikz},$$1

from cells of thickness $$\chi=-\frac{2ikN\mu}{\epsilon_0E}\int_0^{\infty}dz\int dv_z\,W(v_z)\sigma_{ge}(z,v_z)e^{2ikz},$$2 nm (Sargsyan et al., 2019).

Frequency-modulated selective reflection has also been extended to an electric-quadrupole transition. For the cesium $$\chi=-\frac{2ikN\mu}{\epsilon_0E}\int_0^{\infty}dz\int dv_z\,W(v_z)\sigma_{ge}(z,v_z)e^{2ikz},$$3 line at 685 nm near a sapphire window, the measured FMSR signal is

$$\chi=-\frac{2ikN\mu}{\epsilon_0E}\int_0^{\infty}dz\int dv_z\,W(v_z)\sigma_{ge}(z,v_z)e^{2ikz},$$4

and the experiment extracts a collisional broadening coefficient of $$\chi=-\frac{2ikN\mu}{\epsilon_0E}\int_0^{\infty}dz\int dv_z\,W(v_z)\sigma_{ge}(z,v_z)e^{2ikz},$$5 MHz/Torr (Chan et al., 2024). The atom–surface interaction is evidenced, but the quantitative $$\chi=-\frac{2ikN\mu}{\epsilon_0E}\int_0^{\infty}dz\int dv_z\,W(v_z)\sigma_{ge}(z,v_z)e^{2ikz},$$6 extraction remains uncertain because the signal amplitude is only about $$\chi=-\frac{2ikN\mu}{\epsilon_0E}\int_0^{\infty}dz\int dv_z\,W(v_z)\sigma_{ge}(z,v_z)e^{2ikz},$$7 and large modulation depth and long averaging distort the line shape (Chan et al., 2024).

A recent theoretical revision shows that the standard infinite-Doppler approximation in selective-reflection Casimir–Polder spectroscopy can fail badly when the atom–surface interaction is large (Dutta et al., 8 Jul 2025). The full finite-velocity treatment retains the Maxwell–Boltzmann weighting in

$$\chi=-\frac{2ikN\mu}{\epsilon_0E}\int_0^{\infty}dz\int dv_z\,W(v_z)\sigma_{ge}(z,v_z)e^{2ikz},$$8

and shows that for a Rydberg example with $$\chi=-\frac{2ikN\mu}{\epsilon_0E}\int_0^{\infty}dz\int dv_z\,W(v_z)\sigma_{ge}(z,v_z)e^{2ikz},$$9, fitting the exact finite-Doppler spectrum with the infinite-Doppler model yields $e^{2ikz}$0, i.e. a large systematic error (Dutta et al., 8 Jul 2025). For low-lying states the discrepancy is smaller but still nonzero; for the Cs $e^{2ikz}$1 line the old model underestimates $e^{2ikz}$2 by about 10% in the example analyzed (Dutta et al., 8 Jul 2025).

4. Nonlinear selective reflection, dense vapors, and EIT

Selective reflection also supports strongly nonlinear and coherence-based regimes. In pump–probe experiments at a YAG–high-density Rb-vapor interface on the $e^{2ikz}$3 line, with densities $e^{2ikz}$4, the resonant pump reduces both the magnitude and the width of the reflected-probe resonance (Sautenkov et al., 2023). The self-broadening is modeled by

$e^{2ikz}$5

in the linear regime and by

$e^{2ikz}$6

under saturation, with the interpretation that pump depletion of the ground-state population suppresses dipole–dipole self-broadening (Sautenkov et al., 2023). At high pump intensities up to $e^{2ikz}$7, narrow structures appear around the pump frequency (Sautenkov et al., 2023).

A related high-density Rb study analyzes not only $e^{2ikz}$8 but especially its frequency derivative $e^{2ikz}$9, since the derivative has faster-decaying wings and resolves weak nonlinear structure more clearly (Sautenkov et al., 16 Oct 2025). At lower densities, strong pumping produces asymmetric profiles separated by optically saturated dips, interpreted as hole burning in an inhomogeneously broadened line. At the highest density,

$$\bar{\chi}=-\frac{2ik}{\varepsilon_0 E_T}\int_0^\infty dz\,p(z)e^{2ikz},$$0

the derivative spectra split into two nearly symmetric resonances whose separation follows the dressed-state scaling

$$\bar{\chi}=-\frac{2ik}{\varepsilon_0 E_T}\int_0^\infty dz\,p(z)e^{2ikz},$$1

supporting a transition to homogeneously broadened dressed-state splitting (Sautenkov et al., 16 Oct 2025). The empirical fit

$$\bar{\chi}=-\frac{2ik}{\varepsilon_0 E_T}\int_0^\infty dz\,p(z)e^{2ikz},$$2

gives $$\bar{\chi}=-\frac{2ik}{\varepsilon_0 E_T}\int_0^\infty dz\,p(z)e^{2ikz},$$3 and $$\bar{\chi}=-\frac{2ik}{\varepsilon_0 E_T}\int_0^\infty dz\,p(z)e^{2ikz},$$4 GHz at the highest density, but much smaller slopes and positive intercepts at lower densities (Sautenkov et al., 16 Oct 2025).

Electromagnetically induced transparency has likewise been observed in selective reflection from rubidium nanocells. For vapor-column thicknesses from 150 to 1200 nm, EIT in the back-reflected selective-reflection signal, denoted EIT$$\bar{\chi}=-\frac{2ik}{\varepsilon_0 E_T}\int_0^\infty dz\,p(z)e^{2ikz},$$5, is more favorable than EIT in transmission when $$\bar{\chi}=-\frac{2ik}{\varepsilon_0 E_T}\int_0^\infty dz\,p(z)e^{2ikz},$$6 nm (Sargsyan et al., 2024). At $$\bar{\chi}=-\frac{2ik}{\varepsilon_0 E_T}\int_0^\infty dz\,p(z)e^{2ikz},$$7 nm, a weakly spectrally resolved EIT$$\bar{\chi}=-\frac{2ik}{\varepsilon_0 E_T}\int_0^\infty dz\,p(z)e^{2ikz},$$8 is still present, whereas no EIT$$\bar{\chi}=-\frac{2ik}{\varepsilon_0 E_T}\int_0^\infty dz\,p(z)e^{2ikz},$$9 is observed in transmission (Sargsyan et al., 2024). In contrast, in a 50 $\lambda/2\pi$0m microcell, transmission is more effective: EIT$\lambda/2\pi$1 has about 15% contrast and a width of about 9 MHz (Sargsyan et al., 2024). The paper relates the thickness dependence to wall-collision-limited coherence, with

$\lambda/2\pi$2

so the EIT width grows roughly as $\lambda/2\pi$3 in the nanocell regime (Sargsyan et al., 2024).

5. Cholesteric selective reflection and electrically or optically tunable structural color

In cholesteric liquid crystals, selective reflection is a Bragg phenomenon produced by the helical rotation of the director. For a conventional cholesteric, the standard relations are

$\lambda/2\pi$4

for the center wavelength and reflection bandwidth (Xiang et al., 2015). In mixtures of cholesteryl oleyl carbonate (COC) and 5CB, selective-reflection measurements track the helix pitch and show unusual concentration behavior: away from the unwinding region the reciprocal reflection wavelength obeys

$\lambda/2\pi$5

with determination coefficient 0.998, an essentially linear dependence that the authors emphasize as anomalous for a nematic–cholesteric mixture (Lebovka et al., 2013). Near the cholesteric–smectic-A transition, the reflection maximum red-shifts and the band broadens because of critical helix unwinding and pretransitional smectic clustering (Lebovka et al., 2013).

A major extension of cholesteric selective reflection is the field-induced oblique helicoidal, or heliconical, state. In appropriately designed low-$\lambda/2\pi$6, positive-$\lambda/2\pi$7 mixtures, an electric field applied parallel to the helicoidal axis changes both the pitch and the cone angle while preserving the helicoidal axis (Xiang et al., 2015). The field dependence is described by

$\lambda/2\pi$8

and the reflected wavelength remains approximately $\lambda/2\pi$9 (Xiang et al., 2015). Using this mechanism, one study reports continuous tuning of the reflected wavelength from 360 to 1520 nm for the same chemical composition, with a measured absolute reflectance of 41% at 632 nm (Xiang et al., 2015).

Surface anchoring adds a second control mechanism. In oblique-helicoidal cholesteric cells driven at the same average field $|$0, homeotropic anchoring blue-shifts the reflection peak, while planar anchoring red-shifts it (Iadlovska et al., 2018). The explanation is electric-field redistribution caused by the spatially varying dielectric permittivity in the heliconical state near the surfaces. The infinite-slab relation

$|$1

is modified by surface layers whose local $|$2 changes the central-field value and hence the pitch (Iadlovska et al., 2018).

A further development combines electric-field tuning with UV-driven azobenzene photoisomerization. In self-organizing oblique-helicoidal cholesterics doped with photosensitive compounds of slow thermal back-isomerization, UV illumination produces a red shift of selective reflection while the electric field still controls the baseline heliconical state (Mrukiewicz et al., 28 Feb 2025). The reported effect depends strongly on dopant structure: in one chiral photoactive mixture the reflected peak shifts from an initial blue state to about 665 nm after UV, while in another bent-shaped-dopant mixture the shift is only into the green, around 518 nm (Mrukiewicz et al., 28 Feb 2025). The same work states that the molecular structure of the photosensitive materials affects the reflection coefficient, bandwidth, response time to UV irradiation, and tuning range (Mrukiewicz et al., 28 Feb 2025).

6. Other technical uses of the term

The phrase “selective reflection” is also used in several more specialized senses. In thin-film metrology, selective reflection spectroscopy at a sapphire/Cs-film/Cs-vapor interface has been proposed as an in situ probe of metallic film growth (Martins et al., 2013). In that three-layer system the resonant reflectivity change is written

$|$3

where the complex coefficient $|$4 encodes attenuation and phase delay produced by the growing film (Martins et al., 2013). The method clearly tracks spectral evolution during deposition, but under the smooth-film model it does not allow reliable simultaneous extraction of both thickness $|$5 and the van der Waals coefficient $|$6 because the inverse problem is ill-conditioned (Martins et al., 2013).

In superconducting vortex spectroscopy, “selective reflection” refers to spin-selective equal-spin Andreev reflection. For the Sau–Lutchyn–Tewari–Das Sarma heterostructure, a vortex-core Majorana zero mode in the topological phase produces a quantized zero-bias Andreev contribution

$|$7

that is spin selective at the vortex center (Li et al., 2017). The same paper shows that in the trivial phase spin selectivity can still appear at finite bias because of spin-orbit coupling, but it is not topological and vanishes exactly at zero bias due to destructive interference (Li et al., 2017). Experimentally, spin-polarized STM on Bi$|$8Te$|$9/NbSe$|$0 reports that the vortex-center zero-bias conductance is about 14% higher when the tip polarization and the external magnetic field are parallel than when they are anti-parallel, while the effect is absent away from the vortex center and in control samples (Sun et al., 2016).

In chiral superconductors, the term appears again in the form of circular-polarization-selective perfect reflection. For normal incidence along the symmetry axis, the two circular eigenmodes have refractive indices

$|$1

and a sufficiently large optical Hall conductivity can split the plasma edges so that one circular polarization is perfectly reflected while the other is not (Ahn et al., 3 Feb 2025). The paper identifies the condition $|$2 as the requirement for a dissipationless selective-reflection window (Ahn et al., 3 Feb 2025).

In wireless communications, frequency selective reflection of OFDM signals by reconfigurable intelligent surfaces is realized through a time-varying RIS configuration derived from a binary subcarrier-selection mask (Souza et al., 21 Aug 2025). The construction uses

$|$3

and ideally nulls the RIS-assisted response on unselected subcarriers when $|$4 (Souza et al., 21 Aug 2025). In machine learning, “Selective Reflection-Tuning” denotes a teacher–student data-recycling procedure in which a teacher model reflects on instruction-response pairs and a student accepts or rejects the rewritten samples using IFD and r-IFD compatibility scores (Li et al., 2024). These usages are terminologically distinct from optical selective reflection, but they preserve the idea of a response that is selective with respect to a particular channel, polarization, frequency component, spin state, or training sample.