FontCode Steganography

• FontCode steganography is a document watermarking method that encodes data by applying nearly imperceptible perturbations to font glyphs, preserving both visual quality and text content.
• It employs a multi-stage codebook construction using perceptual sampling, CNN-based discrimination, and iterative pruning to generate machine-decodable yet human-indistinguishable glyph variants.
• The approach features robust error correction using Chinese Remainder Theorem encoding and maximum-likelihood decoding, enabling reliable metadata embedding and applications like digital signatures and optical barcodes.

FontCode steganography is a document watermarking and information embedding methodology that encodes user-specified data into text documents by making visually subtle, machine-recognizable perturbations to font glyphs. Information is robustly recoverable from digital vector and raster documents, as well as from photographs of printed hardcopies, while the original text content and aesthetics are preserved (Xiao et al., 2017).

1. Font Manifold and Glyph Generative Model

The FontCode method is grounded in the concept of a continuous "font manifold" for each character, as introduced by Campbell & Kautz (2014). For each character (such as ‘a’), a low-dimensional manifold $\mathcal{M} \subset \mathbb{R}^d$ (with $d=2$ in practice) is learned such that every point $u \in \mathcal{M}$ corresponds to a valid glyph outline $G(u)$. The generative mapping $u \in \mathbb{R}^2 \to G(u)$ constructs vector graphic glyphs (via polyline or spline control points):

$\text{glyph}(u) = \sum_{k=1}^K \phi_k(u) B_k$

where $\{B_k\}$ are learned basis shapes, and $\phi_k(u)$ are smooth weight functions over the 2D manifold. The original font's glyph is located at a reference coordinate $\bar{u}$; encoding proceeds by replacing $\bar{u}$ with a nearby manifold sample $u_i$ selected for information embedding. Such perturbations result in barely perceptible yet systematically altered stroke widths, serifs, or other glyph features.

2. Codebook Construction: Unobtrusive and Machine-Recognizable Perturbations

For each character-font pair, codebooks of machine-recognizable yet perceptually ambiguous glyphs are constructed through a multi-stage process:

• Perceptual Sampling (MTurk Study): Approximately 400 points are sampled in a neighborhood of $\bar{u}$. Human raters perform a two-glyph forced-choice task ("Which is closer to standard font?") and a Bradley-Terry model yields a scalar similarity $s(u)\in[0,1]$. Glyphs with $s(u)\geq \tau_p$ ($\tau_p=0.85$) form the initial candidate set $\mathcal{C}_0$, with this threshold empirically ensuring that perturbed glyphs are indistinguishable from the originals by humans (confirmed in perceptual studies with $p \gg 0.1$ significance).
• Iterative Confusion-Test Pruning: In each round, 100 random candidate glyph pairs are subject to CNN-based discrimination. Edges between glyphs that the CNN cannot distinguish with $\geq95\%$ accuracy are flagged as "confusing," and removed. The maximal clique in the remaining graph forms the updated candidate set $\mathcal{C}$. The process repeats until stabilization.
• Final Machine Verification: For each glyph, a CNN classifier is trained on synthetic and real-photo instances. Glyph samples whose per-glyph classification accuracy falls below $90\%$ are culled.

The resulting codebook for each glyph is a list

$$\mathcal{L} = \lbrace u_0, u_1, \ldots, u_{N-1}\rbrace$$

of $N$ glyph perturbations which are visually indistinguishable but reliably machine-decodable.

3. Embedding User Messages via Glyph Modulation

Given a text document and a bit-string message $M$, FontCode proceeds as follows:

1. The document is parsed, yielding a letter sequence $C_1,\ldots,C_L$, each annotated with its font manifold coordinate $\bar{u}_i$ and bounding box.
2. The text is partitioned into blocks of $n$ letters ($n=5$). For block $t$, the codebook sizes $s_i = |\mathcal{L}_i|$ for each letter are obtained.
3. Select $n$ mutually prime numbers $p_1, ..., p_n$ with $p_i\leq s_i$, maximizing the product of the smallest $k=3$ ($M_t = p_{i_1}p_{i_2}p_{i_3}$).
4. The next $\lfloor \log_2 M_t \rfloor$ bits from $M$ are interpreted as integer $m_t\in [0, M_t)$.
5. For each letter $C_{t,i}$, the codeword component $r_i = m_t \bmod p_i$ determines which codebook glyph $u_{r_i}$ replaces the standard glyph, scaled to the original bounding box.

The message is thus embedded in unobtrusive perturbations of the text's glyphs, blockwise, until an end-of-message marker is inserted.

4. Glyph Recognition and Integer Retrieval

Recovery operates in two modes:

• Pixel-Image Mode: OCR is used to locate each letter and its bounding box; glyphs are cropped, binarized, resized to $200\times200$ pixels, and input to a per-character CNN. The CNN architecture consists of three convolutional layers (kernels: $8\times8$, $5\times5$, $3\times3$), ReLU activation, pooling, followed by two fully connected layers and softmax. The output probabilities $g(u_j, f)$ yield the predicted codebook index $j^* = \arg\max_j g(u_j, f)$.
• Vector-Graphic Mode: The polyline $f$ of each extracted letter is compared against all codebook glyph polylines $u_i$ (rescaled as needed), with L$_2$ distance $d(u_i, f)$ computed; the nearest codebook glyph $i^* = \arg\min_i d(u_i, f)$ is taken as the embedded symbol.

These procedures decode the embedded message via robust glyph recognition in both raster and vector formats, and for scanned or photographed printouts.

5. Error Correction: CRT Coding and Maximum-Likelihood Decoding

Let each $n$-letter block encode a codeword $r = (r_1, ..., r_n)$. Upon recognition, an observed (possibly corrupted) vector $\tilde{r}$ is obtained.

• Chinese Remainder Encoding: Encode $m \in [0, M_t)$ as

$$\phi(m) = (m \bmod p_1, m \bmod p_2, \ldots, m \bmod p_n)$$

The minimum Hamming distance among any two codewords is $H_m = n - k + 1$ (e.g. $3$ for $n=5$, $k=3$).

• Decoding: Compute the candidate integer $\tilde{m}$ using the Chinese Remainder Theorem (CRT):

$$\tilde{m} = \sum_{i=1}^n \tilde{r}_i b_i (P / p_i) \bmod P, \text{ where } P=\prod p_i, \text{ and } b_i (P/p_i) \equiv 1 \ \mathrm{mod}\ p_i$$

If $\tilde{m} < M_t$, accept $\tilde{m}$. Otherwise, Hamming-decode to the nearest valid codeword, correctable up to $\left\lfloor \frac{H_m - 1}{2} \right\rfloor$ errors (i.e. up to 1 misrecognized letter per block by default).

• Maximum-Likelihood Refinement: For ambiguous cases (up to 2 errors per 5-letter block), likelihoods are assigned using the softmax probability (pixel case, $g(u, f)$) or inverse L$_2$ distance (vector case, $1/d(u, f)$). The codeword maximizing likelihood is selected and decoded via CRT. This extends correctability to 2/5 letters per block in practice (40%).

6. Quantitative Evaluation and Performance

• Recognition: Per-glyph CNN accuracy exceeds 90% on real-photo tests.
• Capacity: The average achievable embedding rate is $\sim1.77$ bits per letter in English text. Encoding a 128-bit MD5 hash requires on average 73 letters.
• Robustness: For input images with character height $\geq100$ pixels, end-to-end decoding approaches zero error; accuracy drops for lower resolutions.
• Error Correction: Maximum likelihood decoding yields correction capability for up to 2 out of 5 glyph errors per block (40%), versus 1/5 (20%) for pure CRT/Hamming decoding.
• Runtime: Decoding of 176 letters (35 blocks) executes in 0.89 seconds on a Titan X GPU; encoding requires 7.3 seconds on a single CPU thread.
• Perceptual Studies: MTurk experiments ($N=169,135$) confirm that $\tau_p=0.85$ ensures glyphs are indistinguishable from originals ($p \gg 0.1$ for pairwise discrimination, $p>0.05$ for aesthetic preference). Lower $\tau_p$ (e.g., 0.7) produces noticeable degradation.

7. Demonstrated Applications

FontCode supports multiple applications:

1. Format-Independent Metadata Holder: Embeds metadata (author, date, URL, etc.) within document text, surviving PDF-Word-PNG conversions, printing, and photography—unlike EXIF or XMP metadata.
2. Unobtrusive Optical Barcode: Replaces visual QR barcodes with “invisible” text perturbations. A smartphone app can OCR the text region and decode the embedded information without visible artifacts.
3. Symmetric-Key Encryption: Each codebook is permuted per character using a secret key. Alice and Bob, sharing the permutation, can embed and recover secret information. The keyspace is $>10^{250}$ for lowercase, $>10^{440}$ with uppercase.
4. Document Signature: Supports both symmetric (MD5 hash of the text, embedded using secret permutation) and asymmetric (RSA-signed hash embedded, verifiable via public key) signatures. The signature remains valid through format conversions and reprinting. An extension enables per-segment hashing (∼80-letter blocks) for tamper localization.

These applications illustrate the flexibility of FontCode as an information-carrying layer resilient to perceptual, digital, and physical document transformations (Xiao et al., 2017).