SynNet Architecture Overview

• SynNet is a neural architecture that integrates structure-preserving losses and modular design for cross-modal medical image synthesis and machine comprehension.
• It uses encoder-decoder networks with index-based unpooling and skip connections to maintain anatomical details and enhance feature propagation.
• Empirical evaluations demonstrate improved PSNR and SSIM in medical imaging and effective domain adaptation in QA generation tasks.

SynNet refers to distinct neural architectures introduced for two different domains: cross-modal medical image synthesis (Gunashekar et al., 2018) and transfer learning in machine comprehension (Golub et al., 2017). In both cases, SynNet architectures employ structure-preserving and data-efficient strategies to address limitations of previous methods and to extend performance in data-scarce or domain-shift scenarios.

1. Network Architecture and Design Principles

Medical Image Synthesis

SynNet for medical image synthesis is a fully convolutional network (F-CNN) inspired by U-Net but implements dedicated features to preserve anatomical structure. The encoder consists of sequential blocks incorporating 3×3 convolution, batch normalization, ReLU activation, and max pooling, with pooling indices retained for decoder unpooling. The decoder mirrors the encoder but replaces upsampling with index-based unpooling to restore spatial details and avoid edge blurring. Long-range skip connections concatenate corresponding encoder and decoder feature maps to enhance feature propagation.

A final synthesis block performs 1×1 convolution followed by linear activation for output generation. The architecture is extensible to SISO (Single Input Single Output), MISO (Multiple Input Single Output), and MIMO (Multiple Input Multiple Output) scenarios. In MISO and MIMO modes, multiple parallel encoders process source modalities, with outputs merged at various stages before entering the decoder or generating multi-modal predictions.

Machine Comprehension Transfer

The SynNet for machine comprehension employs a two-stage synthesis approach for transfer to unlabeled domains. The first stage, an answer synthesis module, applies a Bi-LSTM sequence tagger with pre-trained GloVe embeddings and IOB labeling to extract salient answer spans from raw text. The second stage uses a sequence-to-sequence model with attention and copy mechanism ("pointer-generator network") to generate natural language questions, conditioned on both the paragraph and selected answer span. The copy mechanism augments question generation, allowing for direct copying of rare or domain-specific tokens from the context. The two-stage process produces synthetic (Q, A, Paragraph) tuples suitable for data augmentation.

2. Loss Functions and Structure Preservation

Medical Image Synthesis

SynNet introduces a joint structure-preserving loss function, which combines pixel-wise weighted L2 loss, weighted SSIM (Structural Similarity Index Measure) loss, and total variation regularization. Pixel-wise weighting emphasizes critical structural regions, such as edges and small anatomical features, while SSIM loss promotes perceptual and contrast-aware fidelity. Total variation regularization smooths outputs without sacrificing edge precision.

Mathematically,

$$\Im_{overall} = \lambda_1 \Im_{L_2}^{\omega} + \lambda_2 \Im_{SSIM}^{\omega} + \lambda_3 T_{TV} + \lambda_4 R_w$$

with $\omega(x)$ denoting pixel-wise weights, and explicit values $\lambda_1=10$, $\lambda_2=5$, $\lambda_3=0.5$, $\lambda_4=0.0001$ used in evaluation.

Machine Comprehension Transfer

Question synthesis relies on a cross-entropy objective over question tokens, marginalizing latent copy/generate predictors. The decomposition

$P(q, a | p) = P(a | p) \cdot P(q | p, a)$

gives probabilistic structure to the QA generation pipeline.

3. Modes of Operation and Extensibility

Medical SynNet supports configurable input-output mappings:

• SISO for single-modality conversion (e.g., T1→T2 MRI).
• MISO for multi-modal input and single output (e.g., T1 and T1c→T2).
• MIMO for multi-modal, multi-output synthesis (e.g., T1, T1c→T2, Flair).

For machine comprehension, the SynNet can be trained on a source dataset (e.g., SQuAD) and deployed to synthesize QA pairs for another domain with no labeled pairs (e.g., NewsQA). Ablation studies indicate using source data alongside synthetic target QA pairs ("data regularization") maintains generalizability and mitigates overfitting to synthetic noise.

4. Empirical Evaluations

Medical Image Synthesis

On the BRATS dataset (96 training, 24 testing cases across four MRI modalities), SynNet surpasses three competitive baselines—patch-wise CNN, DeconvNet, and U-Net—on PSNR and SSIM metrics. For example:

Model	T1→T2 PSNR	T1→T2 SSIM	T2→T1 PSNR	T2→T1 SSIM
CM-1	13.89	0.63	15.82	0.74
CM-2	19.93	0.54	17.95	0.54
CM-3	16.45	0.52	18.93	0.50
SynNet	20.75	0.65	19.80	0.64

Weighted and joint losses improve SSIM up to 0.86 for T1→T2 synthesis, with gains consistent across all architecture extensions.

Machine Comprehension Transfer

Transferring a SQuAD-trained BIDAF MC model to NewsQA using SynNet yields out-of-domain F1 scores of 44.3% (single model) and 46.6% (ensemble), compared to in-domain oracle performance of 52.3% and vanilla transfer of 39.0%. These results approach supervised lower bounds without using any labeled QA from NewsQA. Mixing synthetic and source QA data is critical; models trained only on synthetic pairs perform worse.

5. Structural Integrity and Practical Impact

Medical SynNet’s structure preservation arises from skip connections, index-based unpooling, and the joint loss. This results in sharper, diagnostically relevant boundaries and improved fidelity in pathology-prone tissues, verified with both quantitative and qualitative results. For machine comprehension, the two-stage synthesis and data regularization allow models to adapt to new vocabularies and answer types, with robust performance across question categories.

Applications include:

• Medical: Modality translation, super resolution, segmentation, and radiotherapy planning.
• NLP: Unsupervised domain adaptation for extractive QA, reduced annotation overhead for new text corpora.

6. Technical Insights, Limitations, and Considerations

Both SynNet architectures demonstrate benefits from architectural modularity, structure-aware losses, and targeted regularization strategies. The medical synthesis framework is constrained by the need for accurate skull-stripped, coregistered data and computational resources for fully convolutional training. MC transfer relies on the quality of synthetic QA pairs and answer selection heuristics; ablation studies suggest enhanced answer selection yields further gains.

A plausible implication is that these structure-preserving and synthesis-driven approaches can catalyze broader progress in domains where labeled data is scarce but cross-modal or cross-domain adaptation is beneficial. However, generalization to radically different domains, or modalities beyond those evaluated, remains to be further characterized.