Use of common-word order syntactic similarity metric for evaluating syllabus coverage of a question paper

Abstract

Syllabuses are used to ensure consistency between educational institutions. A modularized syllabus contains weightages assigned to different units of a subject. Different criteria like Bloom's taxonomy, learning outcomes etc., have been used for evaluating the syllabus coverage of a question paper. But we have not come across any work that focuses on syntactic text similarity evaluation of unit contents with the question contents in order to estimate the syllabus coverage of a question paper. Hence in this paper we address the problem of measuring the syllabus coverage of an examination question paper by using the order based word-to-word syntactic similarity metric. Text preprocessing techniques are used to extract multiple words and its associated locations from textual contents in the question paper and also in the respective syllabus file. Comparison of word order vectors of units with word order vectors of questions results in generation of the corresponding common word pair question vector and common word pair syllabus vector. The common word pair vectors assist in computing the similarity measure between question vector and unit vector, representing the similarity measures in a question-to-unit similarity matrix and selecting the maximal similarity measure among the set of computed common word pair vectors. The maximal similarity measures are used as a guideline in grouping the unit-wise questions, matching its weightage against Syllabus File and evaluating the syllabus coverage of the question paper. The result of syllabus coverage evaluation can be used as a guideline by the subject expert or question paper setter or question paper moderator to revise the questions of examination question paper accordingly.