Over the past two years, the administration of Sana'a University has endeavored to implement an automated Q&A system. At first, certain scientific departments resisted this transition, advocating for the traditional examination system. However, they eventually yielded to the prevailing circumstances.
The motivations of proponents of the automated Q&A system were diverse and not uniform. The university administration was primarily driven by its economic benefits, which saved millions of riyals previously spent each semester on exam papers and the fees of those supervising its conduct.
However, what's strange is that the adoption and implementation of the automated Q&A system occurred quickly and without any discussion about its pros and cons or which scientific subjects it could be applied to. Actually, there was no desire to listen to the opinions of those objecting to its implementation.
Further, the lack of opposition from the scientific departments, which teach large numbers of students, was one of the most important factors that helped in the adoption and implementation of the automated Q&A system. In fact, this system provides convenience for the professors working in these departments, who usually do not receive any financial incentive for manual correction and grading.
On the other hand, supporters of automated questions advocated for this system, justifying its legitimacy and suitability as a means to promote scientific fairness. They argued that it eliminates the subjectivity of the grader, removing any room for personal biases, political leanings, or ideological inclinations of the professor in determining students' results.
This argument is almost the strongest one for the supporters of the automated Q&A system. However, it should be noted that this argument overlooks the existence of a significant number of honest and objective university professors. Therefore, it is not possible to generalize the problem of subjectivity and self-interest to everyone.
Now, after discussing the most important advantages of automated Q&A systems, it is necessary to mention their most significant drawbacks, based on the experiences of many universities that have implemented this system and on a logical analysis of its challenges.
Practically, the automated Q&A system allows students to choose answers based on guessing, which cannot be relied upon in assessing course outcomes. By this, we mean skills in comprehension, cognitive abilities, practical and professional skills, as well as general skills.
Let's take, for example, the impracticality of this system in assessing students' knowledge of mathematics: When the professor provides four or fewer options to solve an equation and the student selects the correct answer, it cannot be ensured that the student chose that answer based on his understanding of the solution method; perhaps they chose that answer through guessing or cheating, especially since this system creates favorable conditions for cheating through signals.
The deficiencies of this system are clearly evident in the humanities as well. For example, how can I know if a student understands Hegel's theory of alienation, Russell's theory of types, the principle of verification in logical positivism, Heidegger's concept of existence, Henri Bergson's concept of intuition, and similar concepts?
Furthermore, in principle, literature cannot be automated either. In this regard, I would like to convey the words of Dr. Abeer Zahra, from the English Language Department at Tishreen University, who opposed the automation, along with all the professors in her department. She states, "What further reinforced our convictions even more is the advancement of students from Damascus University to enroll in our educational qualification program. So, when the interview test was conducted, they were almost unable to speak English or give us answers indicating their academic proficiency, despite their grades exceeding 65%. I confirm that any graduate from the English Language Department at Tishreen University with this grade would be capable and proficient in writing, speaking, and engaging in constructive discussion." ("Exam Automation: A Scientific Farce in Academic Garb," Akkas Al-Sir Website, November 22, 2023).
“It all depends on the educational policy of the state. If its primary goal is to form and create a non-creative individual lacking in knowledge and cognitive skills and ineffective in social and political life, then the automated question system is the suitable system to achieve those objectives.”
Likewise, the automation and auto-correction system is not suitable for language colleges, nor for law, Arabic language, or history colleges. It suffices here to quote what the Dean of the Faculty of Arts at Al-Baath University, Dr. Ghassan Murtada, said: "Automating literature courses' exams eliminates the last opportunity to assess the student's ability to practice language skills properly.
As a matter of fact, the automation of literature subjects, including poetry, drama, novels, and short stories, as well as literary criticism, comparative literature, literary theory, etc., condenses the students' knowledge and confines it to titles, names, and dates, depriving them of delving into the essence of the subjects they study. Here, I wonder: How dare someone automate the subject of short stories, and how did they do that? And how do they automate poetry? Isn't this something strange?" (Ghassan Murtada, “Exam Automation: Between Science and Rumors," Dam Press website, May 23, 2009).
In literature, it is of utmost importance to know the student's ability to express themselves, their knowledge of grammar rules, and their ability to articulate ideas rhetorically. This can never be assessed through automated systems, which only focus on specific information.
Moreover, it has been observed that the complete reliance on automated questions in some universities has deprived graduates of the ability to write correctly and express their thoughts and feelings without grammatical and spelling errors, in addition to the stiffness of their style.
Earlier, we discussed one of the main drawbacks of essay questions, which is the subjective bias of the examiner, who may base their grading on specific ideological or political leanings or may be influenced by prevalent biases, notably regional and sectarian affiliations.
In addition to this flaw, researchers point out several shortcomings, including many students resorting to filling in their answers, which prolongs the time taken for responses to this type of question. Additionally, the use of fancy handwriting and coloring may influence the professor's assessment of their students' results.
Nevertheless, essay exams have essential advantages; as they reveal the student's ability to express ideas in clear language, organize them logically, and demonstrate their knowledge and proficiency in utilizing the methods and techniques of scientific research. This allows for the assessment of his critical and creative thinking skills and familiarity with the art of analysis, ultimately eliminating the possibility of resorting to guesswork.
Based on the preceding discussion, the advantage of essay questions over automated ones becomes apparent. In principle, mitigating the shortcomings of the former one can be achieved through a fusion with objective questions, while also taking into consideration students' participation in scientific seminars and their completion of diverse assignments when assessing the students' final results.
In conclusion, it all depends on the educational policy of the state. If its primary goal is to mold a non-creative individual lacking in knowledge and cognitive skills and ineffective in social and political life, then the automated question system is the suitable system to achieve those objectives.
But if the primary objective of the educational process is to foster a creative personality capable of deeply comprehending scientific knowledge, extensively reproducing it through scientific research, and linking knowledge to real-life applications, then the proposed approach mentioned earlier stands as the appropriate practical means to attain that objective.
Thus, when Yemeni universities reach a high level of scientific, organizational, and administrative development, it becomes possible to abolish written tests and examinations and replace them with oral examinations and assessments, similar to the system practiced in the universities and scientific institutes in the Russian Federation. In these institutions, there is absolutely no room for cheating, and it is possible to assess the student's understanding of various aspects of the curriculum in a very short amount of time.
Accordingly, insisting on implementing the automated Q&A system in Yemeni universities is merely repeating a failed experiment. Its proponents are unwilling to consider the experiences of Arab and foreign universities, which have clearly demonstrated its profound failure in achieving educational and pedagogical objectives.
