SkillRec: A Data-Driven Approach to Job Skill Recommendation for Career Insights

Abstract: Understanding the skill sets and knowledge required for any career is of utmost importance, but it is increasingly challenging in today's dynamic world with rapid changes in terms of the tools and techniques used. Thus, it is especially important to be able to accurately identify the required skill sets for any job for better career insights and development. In this paper, we propose and develop the Skill Recommendation (SkillRec) system for recommending the relevant job skills required for a given job based on the job title. SkillRec collects and identify the skill set required for a job based on the job descriptions published by companies hiring for these roles. In addition to the data collection and pre-processing capabilities, SkillRec also utilises word/sentence embedding techniques for job title representation, alongside a feed-forward neural network for job skill recommendation based on the job title representation. Based on our preliminary experiments on a dataset of 6,000 job titles and descriptions, SkillRec shows a promising performance in terms of accuracy and F1-score.

• The paper introduces SkillRec, which employs word and sentence embeddings with a neural network to map job titles to 589 predefined skills.
• It utilizes advanced techniques like BERT and FastText to encode job titles, achieving superior performance in skill recommendation tasks.
• Experimental results show the BERT+NN configuration achieves the highest accuracy and F1-score compared to classical models.

SkillRec: A Data-Driven Approach to Job Skill Recommendation for Career Insights

Introduction

The accelerating pace of technological advancements and industrial shifts poses significant challenges for workers transitioning across careers, often leading to skill mismatches as industries evolve. The paper "SkillRec: A Data-Driven Approach to Job Skill Recommendation for Career Insights" addresses this issue by presenting the SkillRec system. This system facilitates the identification and recommendation of requisite job skills based on job titles, leveraging data from job descriptions. It utilizes word/sentence embeddings and a feed-forward neural network to map job titles to skill requirements, demonstrating promising performance metrics.

System Architecture and Methodology

SkillRec is built on a modular architecture capable of processing vast arrays of job postings to extract meaningful insights.

Figure 1: System Architecture of Our Proposed Skill Recommendation (SkillRec) System.

Web Collection and Data Pre-processing

The system initiates with a Web Collection Component that scrapes job listings from various portals, capturing over 6,000 unique job posts. Subsequently, the Data Pre-processing Component cleans and standardizes this data, extracting job-relevant features such as titles and descriptions, and mapping them to a predefined list of 589 unique skills derived from online educational resources.

Job Title Representation

For job title representation, SkillRec employs advanced embedding techniques. BERT (Bidirectional Encoder Representations from Transformers) and FastText are utilized to encode job titles into vector representations, significantly enhancing the semantic understanding of the job requirements.

Job Skill Recommendation

These vectorized job titles are input into a feed-forward neural network designed for skill prediction. This model features a two-layer architecture with hidden layers comprising 1280 and 640 nodes respectively, optimized using the AdamW optimizer and employing dropout for regularization. Outputs are interpreted against 589 potential skill sets, each node representing a potential skill.

Experimental Evaluation

The efficacy of SkillRec is validated through a comparison against classical models. Three configurations are assessed: Bag-of-Words, FastText+NN, and BERT+NN. Metrics include accuracy and F1-score, with results showcasing SkillRec's superiority, particularly for the BERT+NN configuration.

Performance Metrics

Algorithm	Accuracy	F1-score
Bag-of-Words	0.4768	0.2887
FastText+NN	0.9728	0.4931
BERT+NN	0.9870	0.4973

BERT+NN achieves the highest accuracy and F1-score, affirming its robustness across various job classification tasks.

Implications and Future Directions

The SkillRec system sets a precedent for using AI-driven insights in career development, reducing the reliance on manual, anecdotal methods for skill identification. Its architecture also indicates potential scalability alongside emerging data sources or shifts in industry demands. Future work could pivot towards dynamic updates of the skill repository to reflect new industry trends or concentrate on augmenting user-specific features, such as prior experience and geographic considerations, to bolster personalized recommendations.

Conclusion

The SkillRec system evidences a promising stride in tackling job skill mismatches through data-driven methodologies. Its design and implementation underscore the potential for AI systems to facilitate smooth career transitions by accurately recommending requisite skills, thereby enhancing workforce adaptability in rapidly changing industrial landscapes.