Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Demystifying Visual Features of Movie Posters for Multi-Label Genre Identification (2309.12022v2)

Published 21 Sep 2023 in cs.AI and cs.CV

Abstract: In the film industry, movie posters have been an essential part of advertising and marketing for many decades, and continue to play a vital role even today in the form of digital posters through online, social media and OTT (over-the-top) platforms. Typically, movie posters can effectively promote and communicate the essence of a film, such as its genre, visual style/tone, vibe and storyline cue/theme, which are essential to attract potential viewers. Identifying the genres of a movie often has significant practical applications in recommending the film to target audiences. Previous studies on genre identification have primarily focused on sources such as plot synopses, subtitles, metadata, movie scenes, and trailer videos; however, posters precede the availability of these sources, and provide pre-release implicit information to generate mass interest. In this paper, we work for automated multi-label movie genre identification only from poster images, without any aid of additional textual/metadata/video information about movies, which is one of the earliest attempts of its kind. Here, we present a deep transformer network with a probabilistic module to identify the movie genres exclusively from the poster. For experiments, we procured 13882 number of posters of 13 genres from the Internet Movie Database (IMDb), where our model performances were encouraging and even outperformed some major contemporary architectures.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. J. A. Wi, S. Jang, and Y. Kim, “Poster-based multiple movie genre classification using inter-channel features,” IEEE Access, vol. 8, pp. 66 615–66 624, 2020.
  2. P. Winoto et al., “The role of user mood in movie recommendations,” Expert Systems with Applications, vol. 37, no. 8, pp. 6086–6092, 2010.
  3. R. M. Lezama et al., “Trailers12k: Evaluating transfer learning for movie trailer genre classification,” arXiv:2210.07983, 2022.
  4. A. Yadav and D. K. Vishwakarma, “A unified framework of deep networks for genre classification using movie trailer,” Applied Soft Computing, vol. 96, p. 106624, 2020.
  5. Q. Hoang, “Predicting movie genres based on plot summaries,” arXiv:1801.04813, 2018.
  6. J. Wehrmann et al., “Self-attention for synopsis-based multi-label movie genre classification,” in Int. FLAIRS Conf., 2018, pp. 236–241.
  7. W.-T. Chu and H.-J. Guo, “Movie genre classification based on poster images with deep neural networks,” in Multimodal Understanding of Social, Affective and Subjective Attributes.   ACM, 2017, p. 39–45.
  8. J. M. Johnson and T. M. Khoshgoftaar, “Survey on deep learning with class imbalance,” Journal of Big Data, vol. 6, pp. 1–54, 2019.
  9. A. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in ICLR, arXiv:2010.11929, 2021.
  10. E. B.-Baruch et al., “Asymmetric loss for multi-label classification,” in CVPR, 2021, pp. 82–91.
  11. G. S. Simões et al., “Movie genre classification with convolutional neural networks,” IJCNN, pp. 259–266, 2016.
  12. X. Yuan et al., “Automatic Video Genre Categorization using Hierarchical SVM,” in ICIP, 2006, pp. 2905–2908.
  13. S. Kar et al., “Folksonomication: Predicting tags for movies from plot synopses using emotion flow encoded neural network,” in ICCL, 2018, pp. 2879–2891.
  14. P. J. Gorinski and M. Lapata, “What’s this movie about? a joint neural network architecture for movie content analysis,” in NAACL: Human Language Technologies, vol. 1.   ACL, 2018, pp. 1770–1781.
  15. I. R. Bribiesca et al., “Multimodal weighted fusion of transformers for movie genre classification,” in Workshop on MAI, 2021, pp. 1–5.
  16. P. C. Bonilla et al., “Moviescope: Large-scale analysis of movies using multiple modalities,” arXiv:1908.03180, 2019.
  17. H. Zhou et al., “Movie genre classification via scene categorization,” in ACM Multimedia, 2010, pp. 747–750.
  18. J. Wehrmann and R. C. Barros, “Movie genre classification: A multi-label approach based on convolutions through time,” Applied Soft Computing, vol. 61, pp. 973–982, 2017.
  19. M. Pobar et al., “Multi-label poster classification into genres using different problem transformation methods,” in CAIP, 2017, pp. 367–378.
  20. N. Gozuacik et al., “Turkish movie genre classification from poster images using convolutional neural networks,” in ELECO, 2019, pp. 930–934.
  21. A. M. Ertugrul et al., “Movie genre classification from plot summaries using bidirectional LSTM,” in ICSC.   IEEE, 2018, pp. 248–251.
  22. V. Battu et al., “Predicting the genre and rating of a movie based on its synopsis,” in PACLIC, 1–3 2018.
  23. R. B. Mangolin et al., “A multimodal approach for multi-label movie genre classification,” Multimedia Tools and Applications, vol. 81, no. 14, pp. 19 071–19 096, 2022.
  24. J. Arevalo et al., “Gated multimodal units for information fusion,” ICLR: Workshop arXiv:1702.01992, 2017.
  25. Z. Rasheed and M. Shah, “Movie genre classification by exploiting audio-visual features of previews,” in ICPR, 2002, pp. 2: 1086–1089.
  26. D. Brezeale and D. J. Cook, “Using closed captions and visual features to classify movies by genre,” in MDM/KDD, 2006, pp. 1–5.
  27. K. Han et al., “A survey on vision transformer,” IEEE Trans. on PAMI, vol. 45, no. 1, pp. 87–110, 2023.
  28. K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” in ECCV, 2016, pp. 630–645.
  29. M. Z. Alom et al., “The history began from AlexNet: A comprehensive survey on deep learning approaches,” arXiv:1803.01164, 2018.
  30. A. Vaswani et al., “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  31. A. Zhang, Z. C. Lipton, M. Li, and A. J. Smola, “Dive into deep learning,” arXiv:2106.11342, 2021.
  32. J. L. Ba et al., “Layer normalization,” arXiv:1607.06450, 2016.
  33. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv:1412.6980, 2014.
  34. L. Rokach, “Ensemble-based Classifiers,” Artificial intelligence review, vol. 33, no. 1, pp. 1–39, 2010.
  35. L. Zahedi et al., “Search algorithms for automated hyper-parameter tuning,” in ICDATA, arXiv:2104.14677, 2021, pp. 1–10.
  36. M.-L. Zhang and Z.-H. Zhou, “A review on multi-label learning algorithms,” IEEE Trans. on KDE, vol. 26, no. 8, pp. 1819–1837, 2013.
  37. G. Huang et al., “Densely connected convolutional networks,” in CVPR, 2017, pp. 2261–2269.
  38. M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in ICML, 2019, pp. 6105–6114.
  39. C. Szegedy et al., “Rethinking the inception architecture for computer vision,” in CVPR, 2016, pp. 2818–2826.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now