2000 character limit reached
Predicting Sustainable Development Goals Using Course Descriptions -- from LLMs to Conventional Foundation Models (2402.16420v2)
Published 26 Feb 2024 in cs.CL
Abstract: We present our work on predicting United Nations sustainable development goals (SDG) for university courses. We use an LLM named PaLM 2 to generate training data given a noisy human-authored course description input as input. We use this data to train several different smaller LLMs to predict SDGs for university courses. This work contributes to better university level adaptation of SDGs. The best performing model in our experiments was BART with an F1-score of 0.786.
- Nlp for sdgs: Measuring corporate alignment with the sustainable development goals. Columbia Business School Research Paper.
- Palm 2 technical report. arXiv preprint arXiv:2305.10403.
- Leslie Mahe Collazo Expósito and Jesús Granados Sánchez. 2020. Implementation of sdgs in university teaching: a course for professional development of teachers in education for sustainability for a transformative action. Sustainability, 12(19):8267.
- Natural language processing for achieving sustainable development: the case of neural labelling to enhance community profiling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8427–8444, Online. Association for Computational Linguistics.
- Unsupervised cross-lingual representation learning at scale.
- CSC - IT Center for Science. 2023. Puhti supercomputer. https://www.csc.fi/en/-/puhti. Accessed: 2023-12-16.
- BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
- Argumentation mining in scientific literature for sustainable development. In Proceedings of the 8th Workshop on Argument Mining, pages 100–111, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Bridging fairness and environmental sustainability in natural language processing. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7817–7836, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. https://spacy.io/. Accessed: 2023-12-16.
- Ching Ting Tany Kwee. 2021. I want to teach sustainable development in my english classroom: A case study of incorporating sustainable development goals in english teaching. Sustainability, 13(8):4195.
- Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In International conference on machine learning, pages 1188–1196. PMLR.
- Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.
- Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
- Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
- OpenAI. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
- Applying sdgs as a systematic approach for incorporating sustainability in higher education. International Journal of Sustainability in Higher Education, 22(6):1266–1284.
- Prompterator: Iterate efficiently towards more effective prompts. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 471–478, Singapore. Association for Computational Linguistics.
- At the intersection of NLP and sustainable development: Exploring the impact of demographic-aware text representations in modeling value on a corpus of interviews. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 2007–2021, Marseille, France. European Language Resources Association.
- Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.