Evaluate the classifier against harder and more representative legitimate advertisements

Evaluate the deceptive-recruitment classifier against harder, more representative negative examples, including legitimate advertisements from small or low-resourced employers, and characterize false-positive rates by employer type and source.

Background

The legitimate comparison group consisted of employers vetted by partner organisations for reputational standing and regulatory compliance. Although this provides a well-characterised contrast, it may make the classification task easier than real-world deployment.

Legitimate low-resource employers may share the same simpler vocabulary and lower-quality imagery that the model associates with deception. The paper therefore identifies evaluation against harder negatives and subgroup-specific false-positive analysis as an unresolved requirement before operational deployment.

References

Consistent with our proof-of-concept framing, we therefore identify evaluation against harder, more representative negatives, with false-positive rates by employer type and source, as a valuable direction for future work before operational deployment; the graduated-response and segmented-threshold safeguards discussed above are intended to manage this risk in practice.

— Detecting Deceptive Recruitment: A Signal-theoretic Machine Learning Framework for Early Identification of Labour Exploitation  (2609.20336 - Siraj et al., 17 Sep 2026) in Section 5.4, Limitations: Negative-class composition