AsseslyAI: AI-Powered Assessment Framework

• AsseslyAI is an AI-powered assessment framework that integrates online lab allocation, unique problem generation, AI-proctored viva, and gamified simulators for engineering lab education.
• It leverages advanced models like CodeLlama-7B with semantic filtering to generate unique, plagiarism-resistant questions and adapt difficulty based on real-time student performance.
• The framework also informs professional and clinical assessments by providing contextual evaluations, interpretability, and dynamic progress analytics for improved learning outcomes.

Searching arXiv for the cited papers and AsseslyAI-related context. AsseslyAI is an AI-powered assessment framework introduced for skill-oriented engineering lab education, where it is designed to address plagiarism, lack of proper lab records, unstructured lab conduction, inadequate execution and assessment, limited practical learning, low student engagement, and absence of progress tracking for both students and faculties (Sharma et al., 27 Sep 2025). The same label, and a spelling variant “AssesslyAI,” also appears in adjacent assessment literature as an example of how AI-based systems could be organized for professional human assessment and for language assessment in dementia, indicating that the name is associated both with a concrete educational pipeline and with a broader family of assessment-oriented AI system designs (Arakawa et al., 2022, Parsapoor et al., 2022).

1. Scope, naming, and assessment context

In its explicit formulation, AsseslyAI is presented as an “Artificial Intelligence-Powered Assessment Framework for Skill-Oriented Engineering Lab Education” (Sharma et al., 27 Sep 2025). Its stated objective is to support practical lab education in computer science through online lab allocation, a unique lab problem for each student, AI-proctored viva evaluations, and gamified simulators. The framework is positioned against a baseline of conventional lab administration and against “traditional automated grading tools,” with emphasis on dataset-driven question generation, adaptive difficulty, plagiarism resistance, and evaluation in a single pipeline.

A spelling variant, “AssesslyAI,” appears in a report that integrates the ideas of “AI for human assessment: What do professional assessors need?” into an AI-powered assessment tool for interview-based evaluation (Arakawa et al., 2022). A second report describes how an AI-powered language-assessment system “such as AsseslyAI” could be organized for dementia-related language impairment assessment (Parsapoor et al., 2022). These usages do not define a single cross-domain product. Rather, they show that the name is used in multiple assessment settings: engineering lab education, professional interviewing, and clinical language assessment.

Across these settings, a recurrent design concern is that assessment is contextual, high-stakes, and difficult to reduce to a single opaque score. In the engineering-lab formulation, this appears as the integration of generation, evaluation, analytics, and engagement into a modular framework (Sharma et al., 27 Sep 2025). In the professional-assessment formulation, it appears as a rejection of black-box suitability scoring in favor of AI that identifies potentially interesting scenes while leaving interpretation to humans (Arakawa et al., 2022).

2. Modular system architecture

AsseslyAI is organized as four core modules—Online Lab Allocation, Unique Lab Problem Generator, AI-Proctored Viva, and Gamified Simulators—tied together by a central Lab Management & Evaluation Engine and a Progress Profiling & Analytics service (Sharma et al., 27 Sep 2025). The high-level layout also includes Client Interfaces and Authentication & Access Control.

Module	Inputs	Outputs
Online Lab Allocation	faculty-defined lab metadata	per-student lab ticket
Unique Lab Problem Generator	lab ticket, faculty keywords, difficulty label	question text, expected answer template, test harness
AI-Proctored Viva	student’s submitted code, question ID	viva questions, live transcript, per-response score
Gamified Simulators	lab performance metrics, profile state	updated profile, interactive mini-games

The Online Lab Allocation module receives topic keywords, difficulty level, viva duration, lab mode, description, and instructions. Upon faculty “Create Lab,” it writes metadata to the central engine, triggers the Unique Lab Problem Generator for each enrolled student, and grants time-bound access via the Student Dashboard (Sharma et al., 27 Sep 2025).

The Unique Lab Problem Generator uses a fine-tuned CodeLlama-7B model to synthesize a unique, code-rich programming question per student. It applies semantic filtering with an SBERT-based similarity threshold $T_{\text{dedupe}} = 0.6$ to enforce non-repetition, and it logs generation metadata such as seed and timestamp to prevent re-generation (Sharma et al., 27 Sep 2025).

The AI-Proctored Viva module contains an Automatic Speech Recognition module, exemplified by Wav2Vec2, a Natural Language Understanding module using SBERT embeddings and an intent classifier, and a Scoring Rubric engine based on a pre-compiled question–answer rubric from faculty inputs. After code submission, it schedules a viva session, streams audio to ASR, routes transcribed answers to a rubric matcher, and aggregates scores and feedback (Sharma et al., 27 Sep 2025).

The Gamified Simulators module consumes time to completion, code correctness, viva score, and profile state variables such as XP and badges. It outputs updated profile state, leaderboard position, and mini-games tied to key ML/AI concepts, including a hyperparameter tuning puzzle and neural network building blocks (Sharma et al., 27 Sep 2025).

3. Dataset construction, fine-tuning, and evaluator calibration

The framework is trained and calibrated on a dataset of 10,000 synthetic question–answer pairs targeting ML/AI lab topics (Sharma et al., 27 Sep 2025). Questions are generated through a hybrid pipeline involving Python scripts and ChatGPT, answers are produced using Perplexity AI, and the resulting material undergoes manual faculty review. Topic distribution is reported as approximately 40% classical ML, 40% deep learning, and 20% optimization and NLP. Each record contains a Python code answer with conceptual commentary, while theoretical questions are embedded as code-comment prompts. Difficulty labels are distributed as Easy 30%, Medium 50%, and Hard 20%, and each instance carries both marksAI and marksFaculty in the range 0–100 for calibration.

For question generation, the backbone is CodeLlama-7B-Instruct with LoRA adapters (Sharma et al., 27 Sep 2025). The reported hyperparameters are epochs $= 3$ with an initial 1-epoch sanity check, batch size/device $= 1$, gradient accumulation $= 8$, learning rate $= 1 \times 10^{-4}$ with AdamW, FP16 with 4-bit quantization, LoRA $r = 8$, $\alpha = 32$, dropout $= 0.05$, and maximum sequence length $= 512$, extended to 1024. The loss is standard token-level cross-entropy.

Post-generation filtering is semantic rather than purely lexical. For candidate question pairs, SBERT embeddings $E(Q)$ and $= 3$0 are computed, and a candidate is discarded if

$= 3$1

Evaluator calibration is handled separately. The embedding model is Sentence-BERT all-MiniLM-L6-v2 with 384 dimensions, and the regressor is XGBoost with $= 3$2, $= 3$3, $= 3$4, $= 3$5, and $= 3$6 (Sharma et al., 27 Sep 2025). The loss is mean squared error, and 5-fold cross-validation yields RMSE $= 3$7 and $= 3$8.

4. Dynamic generation, adaptive difficulty, plagiarism resistance, and viva scoring

The dynamic question-generation pipeline iterates over the class roster, builds a prompt from faculty keywords and difficulty, generates a raw question with CodeLlama, checks semantic similarity against older questions, retries generation up to a bounded number of attempts when the similarity threshold is exceeded, post-processes the accepted question, and saves it student-wise (Sharma et al., 27 Sep 2025). The framework therefore operationalizes per-student uniqueness as a routine property of the generation loop rather than as a post hoc correction.

Difficulty adjustment is modeled as a real-valued scalar $= 3$9, with student performance $= 1$0 expressed as normalized score. The update rule is

$= 1$1

where $= 1$2 is the desired proficiency and $= 1$3 is a step size (Sharma et al., 27 Sep 2025). If $= 1$4, the next question becomes slightly harder; otherwise it becomes easier.

Plagiarism resistance is implemented through several mechanisms. Unique question enforcement uses the SBERT cosine similarity threshold $= 1$5 together with Winnowing fingerprinting on code scaffolds to detect near-duplicate templates (Sharma et al., 27 Sep 2025). Copy-paste detection uses cosine similarity of code embeddings,

$= 1$6

and normalized Levenshtein distance,

$= 1$7

A submission is flagged if cosine similarity exceeds $= 1$8 or $= 1$9 (Sharma et al., 27 Sep 2025).

The AI-proctored viva converts student speech to text through Wav2Vec2, aligns responses to rubric points using SBERT-based intent and keyphrase extraction, and scores responses by weighted embedding similarity (Sharma et al., 27 Sep 2025). The scoring function is given as

$= 8$0

with full credit if $= 8$1 and partial credit if $= 8$2. The evaluation criteria are conceptual accuracy at 40%, domain terminology use at 30%, and reasoning coherence at 30%.

5. Gamified simulators and reported empirical performance

The gamified component is built around three design principles: flow theory, immediate feedback, and progression mechanics (Sharma et al., 27 Sep 2025). Flow theory is instantiated by balancing challenge and skill through adaptive difficulty. Immediate feedback appears as real-time hints in mini-games, including a gradient-descent maze. Progression mechanics are implemented through XP, badges, and levels. Mastery reinforcement occurs by unlocking progressively complex simulators only when conceptual checkpoints are passed, such as successfully tuning a small neural network before tackling a convolutional filter puzzle.

Engagement is monitored through session duration per lab, completion rate of mini-games, badge acquisition count, and leaderboard rank changes (Sharma et al., 27 Sep 2025). These measurements tie the gamified subsystem back to the Progress Profiling & Analytics service and faculty dashboards.

Validation metrics reported for the full framework include question–answer semantic similarity, non-repetitiveness, assessment accuracy, and evaluator regression performance (Sharma et al., 27 Sep 2025). The average question–answer semantic similarity is $= 8$3 with $= 8$4 over 10,000 pairs. Duplicate rate is reported as less than 5% after semantic filtering, compared with greater than 30% for template baselines. Assessment accuracy is summarized by Pearson $= 8$5, Spearman $= 8$6, and Cohen’s $= 8$7. Evaluator regression reproduces the earlier RMSE $= 8$8 marks and $= 8$9.

The experimental setup involves approximately 200 students across two sections of an ML lab course, with a comparison baseline of a static template engine plus dynamic test cases (Sharma et al., 27 Sep 2025). Each student completed six labs over eight weeks, with one section using AsseslyAI and the other the baseline. The reported gains are +22% in hands-on skill assessment, +18% in conceptual mastery, +35% in engagement as measured by session time, and 2 plagiarism incidents flagged versus 12 in the baseline condition.

6. Human-assessment lineage: observation, interpretability, and assessor confidence

A related design trajectory appears in interview-based professional assessment, where AI is introduced not as a direct judge but as a decision-support mechanism for professional assessors (Arakawa et al., 2022). The motivating challenges are that interview review is time-consuming and mentally taxing, assessors record 10–15 minute sessions and may spend up to 30 minutes reviewing each video, human subjectivity and cognitive bias can reduce fairness and consistency, and a black-box suitability score is neither trusted nor defensible when clients ask for rationale.

The derived requirements are separation of observation and judgment, unsupervised data-driven cue extraction, feature-level interpretability, and a lightweight in-context UI (Arakawa et al., 2022). The implemented algorithm is an online Gaussian Mixture Model over multimodal behavioral windows, with

$= 1 \times 10^{-4}$0

and outlier score

$= 1 \times 10^{-4}$1

Behavioral features include facial keypoints, body pose, 3D head pose, and gaze direction.

Interpretability is realized by a per-modality contribution score $= 1 \times 10^{-4}$2, obtained by masking one modality block of $= 1 \times 10^{-4}$3, recomputing likelihood, and measuring the reduction in outlierness (Arakawa et al., 2022). The interface prototype places pins on the timeline for the top six anomaly windows, distinguishes top 3 from ranks 4–6 by color, provides a tooltip identifying the main contributing modality, and supports click-to-jump to the flagged segment.

In the reported study with two assessors and twenty videos, assessor-to-assessor agreement on selected cues was 52.5%, while algorithm recall against each assessor’s list was 38% at top 10 and 51% when top 15 were considered (Arakawa et al., 2022). A second study with six assessors found enhanced objectivity and confidence, no overreliance or loss of trust, requests for expansion toward paralinguistic features, and interest in using flagged scenes as evidence during feedback sessions. A plausible implication for systems discussed under the AsseslyAI label is that interpretability and explicit separation of observation from judgment remain central when assessment is highly contextual.

7. Language-assessment extension: clinical workflow, biomarkers, and deployment considerations

A second adjacent formulation describes an AI-powered language-assessment system for dementia in older adults and explicitly frames it as a system “such as AsseslyAI” (Parsapoor et al., 2022). The architecture is organized into five major stages: data collection, preprocessing, feature extraction, model training and evaluation, and deployment. The study recruits participants with dementia and matched healthy controls, with $= 1 \times 10^{-4}$4 dementia and $= 1 \times 10^{-4}$5 controls, and elicits Picture Description and Story Recall tasks through both telephone-line and web-browser recordings.

Feature extraction spans linguistic and acoustic biomarkers (Parsapoor et al., 2022). Linguistic measures include Brunet’s Index, Honoré’s Statistic, Flesch–Kincaid Grade Level, Flesch Reading Ease, propositional density, content density, and embedding-based coherence and tangentiality measures. Acoustic features include 15 MFCCs and delta coefficients, Log-Mel filterbank energies, 8 Line Spectral Pairs and deltas, $= 1 \times 10^{-4}$6, jitter, shimmer, loudness and deltas, voicing probability, pause durations, and speech or articulation rate, summarized into descriptive statistics producing approximately 296 acoustic predictors.

The evaluated binary classifiers include SVM, Logistic Regression, Decision Tree, Random Forest, Extremely Randomized Trees, kNN, LDA, and, if data permits, CNN, with hyperparameter tuning by grid search and 3-fold cross-validation using micro-averaged F1 (Parsapoor et al., 2022). Reported results indicate that Picture Description yields richer predictive markers than Story Recall, with acoustic-only Random Forest at approximately $= 1 \times 10^{-4}$7 and linguistic Extremely Randomized Trees at approximately $= 1 \times 10^{-4}$8, while Story Recall underperforms due to fewer tokens and sentences. Recording media also matter: phone-based data produce mean $= 1 \times 10^{-4}$9 versus web-based $r = 8$0, and phone acoustic ET yields $r = 8$1 versus web ET $r = 8$2.

The same report emphasizes reliability, validity, and fairness safeguards: one-way ANOVA with Tukey’s post-hoc tests, effect sizes such as Cohen’s $r = 8$3, possible future validation against MMSE and MoCA through Intraclass Correlation Coefficient and Pearson’s $r = 8$4, exclusion of sensitive attributes such as gender and education, external-cohort testing, and explainable models through tree-based feature importances and SHAP values (Parsapoor et al., 2022). Deployment recommendations include containerizing preprocessing, feature extraction, and model inference in microservices, exposing RESTful APIs to a front-end application, monitoring live performance drift, recalibrating with new labeled data through continual learning, and logging uncertainty to flag borderline cases for expert review. These elements indicate how the AsseslyAI label can be extended from educational assessment into clinically oriented language assessment while preserving a strongly evaluation-centric system design.