Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Hierarchical Knowledge Distillation on Text Graph for Data-limited Attribute Inference (2401.06802v1)

Published 10 Jan 2024 in cs.CL, cs.LG, and cs.SI

Abstract: The popularization of social media increases user engagements and generates a large amount of user-oriented data. Among them, text data (e.g., tweets, blogs) significantly attracts researchers and speculators to infer user attributes (e.g., age, gender, location) for fulfilling their intents. Generally, this line of work casts attribute inference as a text classification problem, and starts to leverage graph neural networks (GNNs) to utilize higher-level representations of source texts. However, these text graphs are constructed over words, suffering from high memory consumption and ineffectiveness on few labeled texts. To address this challenge, we design a text-graph-based few-shot learning model for attribute inferences on social media text data. Our model first constructs and refines a text graph using manifold learning and message passing, which offers a better trade-off between expressiveness and complexity. Afterwards, to further use cross-domain texts and unlabeled texts to improve few-shot performance, a hierarchical knowledge distillation is devised over text graph to optimize the problem, which derives better text representations, and advances model generalization ability. Experiments on social media datasets demonstrate the state-of-the-art performance of our model on attribute inferences with considerably fewer labeled texts.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Attriguard: A practical defense against attribute inference attacks via adversarial machine learning. In 27th USENIX Security Symposium (USENIX Security 18), pages 513–529, 2018.
  2. Adversary for social good: Leveraging attribute-obfuscating attack to protect user privacy on social networks. In International Conference on Security and Privacy in Communication Systems, pages 710–728. Springer, 2022a.
  3. Turning attacks into protection: Social media privacy protection using adversarial attacks. In Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pages 208–216. SIAM, 2021.
  4. Adversary for social good: Leveraging adversarial attacks to protect personal attribute privacy. arXiv preprint arXiv:2306.02488, 2023a.
  5. α𝛼\alphaitalic_α-satellite: An ai-driven system and benchmark datasets for hierarchical community-level risk assessment to help combat covid-19. arXiv:2003.12232, 2020.
  6. Chung-Ying Lin. Social reaction toward the 2019 novel coronavirus (covid-19). Social Health and Behavior, 2020.
  7. Adversarial classification on social networks. In AAMAS, 2018.
  8. Attribute inference attacks in online social networks. TOPS, 2018.
  9. Attriinfer: Inferring user attributes in online social networks using markov random fields. In WWW, 2017.
  10. Alex Graves. Long short-term memory. In Supervised Sequence Labelling with Recurrent Neural Networks. 2012.
  11. Transformer-xl: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860, 2019.
  12. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
  13. Graph convolutional neural networks for web-scale recommender systems. In SIGKDD, 2018.
  14. Pseudo-labeling with graph active learning for few-shot node classification. In IEEE International Conference on Data Mining. IEEE, 2023b.
  15. Hover: Homophilic oversampling via edge removal for class-imbalanced bot detection on graphs. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 3728–3732, 2023.
  16. Hierarchical graph neural network for patient treatment preference prediction with external knowledge. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 204–215. Springer, 2023c.
  17. Semi-supervised user profiling with heterogeneous graph attention networks. In IJCAI, 2019.
  18. A social spatio-temporal graph convolutional neural network for human trajectory prediction. In CVPR, 2020.
  19. Graph convolutional networks for text classification. In AAAI, 2019.
  20. Be more with less: Hypergraph attention networks for inductive text classification. arXiv preprint arXiv:2011.00387, 2020.
  21. Hierarchical heterogeneous graph representation learning for short text classification. arXiv:2111.00180, 2021.
  22. Every document owns its structure: Inductive text classification via graph neural networks. arXiv preprint arXiv:2004.13826, 2020.
  23. Text level graph neural network for text classification. arXiv preprint arXiv:1910.02356, 2019.
  24. Heterogeneous graph attention networks for semi-supervised short text classification. In EMNLP-IJCNLP, 2019.
  25. Distilling knowledge on text graph for social media attribute inference. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2024–2028, 2022b.
  26. Knowledge distillation on cross-modal adversarial reprogramming for data-limited attribute inference. In Companion Proceedings of the ACM Web Conference 2023, pages 65–68, 2023d.
  27. GloVe: Global vectors for word representation. In EMNLP), 2014.
  28. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.
  29. Sentence-BERT: Sentence embeddings using siamese bert-networks. In EMNLP, 2019.
  30. Few-shot learning with graph neural networks. arXiv preprint arXiv:1711.04043, 2017.
  31. Neural message passing for quantum chemistry. In ICML, 2017.
  32. Learning to propagate labels: Transductive propagation network for few-shot learning. arXiv preprint arXiv:1805.10002, 2018.
  33. Enhancing robustness of graph convolutional networks via dropping graph connections. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14–18, 2020, Proceedings, Part III, pages 412–428. Springer, 2021.
  34. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
  35. Dynamic distillation network for cross-domain few-shot recognition with unlabeled data. Advances in Neural Information Processing Systems, 34:3584–3595, 2021.
  36. A latent variable model for geographic lexical variation. In Proceedings of the 2010 conference on empirical methods in natural language processing, pages 1277–1287, 2010.
  37. Effects of age and gender on blogging. In AAAI spring symposium: Computational approaches to analyzing weblogs, 2006.
  38. Learning to propagate labels: Transductive propagation network for few-shot learning. In International Conference on Learning Representations, 2019.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Quan Li (66 papers)
  2. Shixiong Jing (1 paper)
  3. Lingwei Chen (8 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now