Omission-to-Modification Error Ratio

• Omission-to-Modification Error Ratio is a quantitative measure comparing omitted true ties (false negatives) with incorrectly reported ties (false positives) in network data.
• It is employed in ROC-based aggregation methods where adjusting thresholds and weighting factors balances omission and modification errors for optimal network recovery.
• Empirical studies in CSS networks show O:M ratios ranging from 4 to 20, highlighting the trade-off between liberal and conservative reporting in sparse social structures.

The Omission-to-Modification error ratio (O:M ratio) arises in the context of cognitive social structure (CSS) network studies, where individuals report not only their own direct social ties but also their perceptions of ties among all others in a bounded network. The O:M ratio serves as a quantitative metric that captures the trade-off between omission errors (false negatives, where actual ties are missed) and modification errors, also called commission errors (false positives, where non-ties are incorrectly reported as present). Used in conjunction with ROC-curve based aggregation methods, the O:M ratio enables explicit calibration of network estimates by controlling the relative emphasis placed on the minimization of each error type (Yenigun et al., 2016).

1. Formal Definition

The true relational structure is represented by a directed adjacency matrix $A=(A_{ij})$ with size $N \times N$, where $A_{ij}=1$ if a true tie exists from $i$ to $j$ and $0$ otherwise. The estimated or perceived structure, denoted $\hat{A}=(\hat{A}_{ij})$, may be inferred via aggregation, or may refer to a single respondent’s report. Using the indicator $I\{\cdot\}$ that is $1$ if its argument is true and 0 otherwise, the error rates are defined as:

• Omission error rate $E_O$ (false negatives):

$N \times N$0

This quantifies the proportion of true ties missed in estimation.

• Modification (commission) error rate $N \times N$1 (false positives):

$N \times N$2

This reflects the proportion of non-ties incorrectly categorized as ties.

• O:M error ratio:

$N \times N$3

This ratio encapsulates the relative prevalence of omission versus modification errors in any given network estimation scenario.

2. Role in ROC–Based Aggregation and Weighting

In ROC-curve based CSS aggregation, a threshold parameter $N \times N$4 represents the minimum number of respondents required to agree on a tie for it to be considered present. For each $N \times N$5, $N \times N$6 and $N \times N$7 are calculated, generating an empirical ROC curve plotting true positive rate (TPR$N \times N$8) against false positive rate (FPR$N \times N$9). The classical ROC criterion minimizes the Euclidean distance to the ideal $A_{ij}=1$0 point, corresponding to

$A_{ij}=1$1

which treats omission and commission errors equally ($A_{ij}=1$2).

The introduction of an explicit weighting factor $A_{ij}=1$3 generalizes this to

$A_{ij}=1$4

where $A_{ij}=1$5 corresponds to desired O:M emphasis ($A_{ij}=1$6 can also be denoted $A_{ij}=1$7). Selecting $A_{ij}=1$8 up-weights the cost of modification/commission errors, leading the optimization to favor thresholds that reduce false positives, often at the expense of increased omissions. The procedure thus allows the analyst to choose or justify a particular O:M trade-off directly in accordance with substantive considerations or network sparsity (Yenigun et al., 2016).

3. Conceptual and Empirical Patterns in CSS Work

CSS studies characteristically involve networks with low density ($A_{ij}=1$9) and a large imbalance between the number of possible ties and non-ties. Empirically, omission error rates ($i$0) are observed between 0.54 and 0.72, while modification error rates ($i$1) range from 0.03 to 0.14, yielding raw O:M ratios in the 4 to 20 range across studied datasets.

There is a strong negative correlation between $i$2 and $i$3 at the respondent level, indicating a trade-off: respondents who are "liberal" in their perceptions tend toward low omission but high commission errors; "conservative" respondents exhibit the reverse pattern. This supports the methodological need for explicit O:M balancing in aggregation (Yenigun et al., 2016).

4. O:M Ratio in Network Estimation Algorithm

The ROC-based threshold method for network aggregation (RTM) uses the O:M ratio via the weighting parameter $i$4 to make the error trade-off both explicit and data-driven:

1. Randomly sample $i$5 CSS slices; compute their average density $i$6.
2. Set $i$7 (or another value according to analytic priorities).
3. For each candidate threshold $i$8, calculate $i$9, $j$0, and $j$1.
4. Select $j$2, and aggregate the network using this threshold.

In sparse networks, the recommended weight $j$3 amplifies the importance of reducing false positives, which are disproportionately likely amidst a majority of non-ties.

5. Numerical and Simulation Results

Across five canonical CSS datasets (network size $j$4–$j$5, density $j$6–$j$7), observed mean $j$8 values are $j$9–$0$0 and $0$1 values are $0$2–$0$3, with O:M ratios generally between 4 and 20. An illustrative example ("High Tech Managers", $0$4, $0$5) demonstrates how varying $0$6 affects the O:M ratio:

Threshold $0$7	$0$8	$0$9	O:M
1	0.295	0.083	0.28
4	0.034	0.667	19.6

Correlation with the true network structure increases under the $\hat{A}=(\hat{A}_{ij})$0-weighted ROC choice (e.g., $\hat{A}=(\hat{A}_{ij})$1) versus an unweighted choice ($\hat{A}=(\hat{A}_{ij})$2). Large-scale simulations confirm that using $\hat{A}=(\hat{A}_{ij})$3 in the ROC minimization yields robust recovery, performing comparably or better than adaptive thresholding which constrains only commission errors.

6. Guidelines and Applied Selection

Choice of $\hat{A}=(\hat{A}_{ij})$4 should be context-driven:

• If false positives (commission errors) are much costlier, set $\hat{A}=(\hat{A}_{ij})$5 to minimize $\hat{A}=(\hat{A}_{ij})$6 further.
• If false negatives (omission errors) are to be avoided, set $\hat{A}=(\hat{A}_{ij})$7.
• If no clear preference exists, use $\hat{A}=(\hat{A}_{ij})$8 (classical ROC methodology).

The recommended approach is $\hat{A}=(\hat{A}_{ij})$9 in sparse organizational networks, balancing the error rates for superior network recovery (Yenigun et al., 2016).

7. Worked Example

For a toy network with $I\{\cdot\}$0, $I\{\cdot\}$1 sampled slices, and observed counts as follows:

$I\{\cdot\}$2 (threshold)	False Positives	False Negatives	$I\{\cdot\}$3	$I\{\cdot\}$4	$I\{\cdot\}$5	$I\{\cdot\}$6 (for $I\{\cdot\}$7=5)
1	2	1	0.333	0.25	0.417	1.667
2	1	2	0.167	0.5	0.528	1.305
3	0	3	0	0.75	0.75	0.75

With equal weighting ($I\{\cdot\}$8), $I\{\cdot\}$9 minimizes $1$0. With $1$1, corresponding to high stringency, $1$2 minimizes $1$3. This formalism directly operationalizes the trade-off between omission and commission errors within ROC-guided network estimation (Yenigun et al., 2016).