Open Category Submission

262 – Towards Automatic Arabic Test Scoring: Leveraging ChatGPT for Prompt Design

ChatGPT possesses a powerful large language model
that relies on deep learning technology. It excels at understanding
natural language and context, enabling it to generate answers
to questions and inquiries. However, evaluating and grading
high school Arabic Achievement tests in Saudi Arabia requires
human interventions by providing correct test answers into the
tests platforms. In this paper, we propose a web-based system
(AutoGrader), which includes a framework that automatically
grades the students’ tests without human interventions. The
framework relies on ChatGPT and Term Frequency-inverse Doc-
ument Frequency (TF-IDF) algorithm. The framework compares
students’ answers with those generated by ChatGPT using the
TF-IDF similarity measure and then displays students’ grades
rapidly. The framework is evaluated using students’ tests. The
framework is efficient and effective, which can be used to improve
and enhance educational assessments and experiences.

Overview Of The Project
This project enables students to generate and evaluate exams using
artificial intelligence. The teacher can compose questions to assess the students. Once the questions are written, students can proceed to take the test. Upon completion, their answers are compared with those generated by ChatGPT, which serves as a valuable feature for the project. Furthermore, the teacher and the student can view the scores once the test is completed. Problem Statment
Many exams today are cumbersome for teachers to grade, often consuming a lot of time. Regrettably, students are left waiting for extended periods to receive their grades. In light of this issue, we conceived the idea for this project. Proposed System
We are creating a website that utilizes Al (ChatGPT) to facilitate student achievement testing

Stream3

A framework using ChatGPT and the TF-IDF algorithm has been developed to achieve success. This framework aims to compare target customers with ChatGPT ads using TF-IDF similarity scores. The teacher generates questions, and the student answers them. The program uses the TF-IDF algorithm to attract customers who then attract ChatGPT. Initially, all students are counted, and they submit their predictions to the teacher. This project includes six main components: the teacher, the student, the test, the database, similarity checking, and the TF-IDF algorithm. It does not allow for a centralized storage of data and answers, resulting in organized data and results retrieval.

The execution speed of the algorithms for the test sections was evaluated by comparing the two ChatGPT models. The results showed that ChatGPT-3.5 achieved 60% accuracy, while ChatGPT-4.0 achieved 85% accuracy, reflecting its superiority in understanding questions and providing accurate answers. The speed of the TF-IDF algorithm was tested in four sections: chemistry, biology, mathematics, and physics. It was observed that increasing the number of answers affected execution speed, with times increasing as the number of answers increased, indicating that the system may struggle to process larger amounts of data.

Innovation

1. Instant Feedback Mechanism: By comparing student answers with those generated by ChatGPT, the project provides immediate feedback, significantly reducing the waiting time for grades and enhancing the learning process.

2. Dynamic Question Bank: The ability for teachers to create and modify questions on the fly ensures that assessments remain relevant and aligned with current educational standards and practices.

3. Data-Driven Insights: The system can collect and analyze performance data, providing educators with valuable insights into student learning patterns, enabling targeted interventions and curriculum improvements.
Creativity

1. Engaging User Experience: Designing an intuitive and visually appealing interface for both students and teachers can enhance engagement, making the testing process more enjoyable and less stressful.

2. Gamification Elements: Introducing gamification features, such as badges for achievements, leaderboards, or interactive quizzes, can motivate students and create a more dynamic learning environment.

By harnessing innovation and creativity, this project not only addresses current challenges in educational assessment but also enriches the learning experience for students and teachers alike.

Scalability

1. User Growth: The system can accommodate an increasing number of users, including both students and teachers. As more educational institutions adopt the platform, the infrastructure can be scaled to support this growth.

2. Question Bank Expansion: The ability to add more subjects and question types allows the project to cater to diverse curricula. This expansion can attract more users and enhance the platform’s utility.

3.Integration of Additional Features: New functionalities, such as analytics dashboards for teachers, adaptive testing mechanisms, or personalized study recommendations, can be integrated without disrupting current operations.
Sustainability

1.Efficient Grading: Automating the grading process reduces the time teachers spend on evaluations, allowing them to focus on instructional quality and student engagement, contributing to better educational outcomes.

2. Continuous Feedback Loop: Incorporating mechanisms for user feedback will allow for iterative improvements, ensuring that the platform evolves with the needs of students and educators.

3. Data Privacy and Security: Implementing robust data protection measures will ensure compliance with educational regulations and build trust among users, which is crucial for long-term sustainability.

Our system provides an efficient and easy-to-use mechanism for student assessment, enabling quick access to assessments. It also helps teachers reduce the time spent marking tests, allowing them to focus on providing educational support to their students. The system ensures that all students receive fair assessments based on their actual performance, reducing bias in the assessment process.

To measure social impact, we conduct surveys to assess teacher and student satisfaction with the system. To keep pace with evolving needs, we continually update the system to keep pace with the latest technologies to ensure its continued effectiveness.

Personal Information Collection Statement (PICS):
1. The personal data collected in this form will be used for activity-organizing, record keeping and reporting only. The collected personal data will be purged within 6 years after the event.
2. Please note that it is obligatory to provide the personal data required.
3. Your personal data collected will be kept by the LTTC and will not be transferred to outside parties.
4. You have the right to request access to and correction of information held by us about you. If you wish to access or correct your personal data, please contact our staff at lttc@eduhk.hk.
5. The University’s Privacy Policy Statement can be access at https://www.eduhk.hk/en/privacy-policy.

• I have read and agree to the competition rules and privacy policy.