A syllabus-fairness measure for evaluating open-ended questions

Abstract

A modularized syllabus containing weightages assigned to different units of a subject proves very useful to both teaching as well as student community. Different criteria like Bloom's taxonomy, learning outcomes etc., have been used for evaluating the fairness of a question paper. But we have not come across any work that focuses on unit-weightages for computing the syllabus fairness. Hence in this paper we address the problem of evaluating the syllabus-fairness of open-ended questions of an examination question paper by analyzing the questions on different criteria. Text mining techniques are used to extract keywords from textual contents in the syllabus file and also in the question paper. Similarity Coefficient is used to compute the similarity between question content and syllabus content. Similarity measure is identified by computing the similarity matrix between question vectors and syllabus vectors. The similarity matrix is used as a guideline in grouping the unit-wise questions; matching its weightage against Syllabus File and evaluating the syllabus fairness of the question paper. The result of syllabus fairness evaluation can be used as a measure by the subject expert or question paper setter or question paper moderator to revise the questions of examination question paper accordingly.