Open Category Submission

352 – MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection

MathAgent is an innovative AI-powered framework designed to revolutionize mathematical error detection in educational settings. Unlike traditional methods relying on manual review or simplistic AI models, MathAgent leverages a novel Mixture-of-Math-Agent architecture. This framework decomposes the complex task of multimodal error detection into three specialized agents: an Image-Text Consistency Validator, a Visual Semantic Interpreter, and an Integrative Error Analyzer. The Image-Text Consistency Validator ensures semantic alignment between visual and textual problem elements. The Visual Semantic Interpreter extracts structured mathematical expressions from diagrams and images. Finally, the Integrative Error Analyzer correlates all inputs, including student solutions, to pinpoint error locations and categorize misconceptions. This specialized approach enables MathAgent to accurately process multimodal mathematical content, including diagrams, equations, and text, leading to more precise error identification and categorization. This enhanced accuracy facilitates targeted feedback and personalized learning interventions, significantly improving student understanding and learning outcomes.

The inspiration for MathAgent stems from the limitations of existing mathematical error detection methods. Manual review is costly and unscalable, while current AI models struggle with the nuances of multimodal mathematical content. Our hypothesis is that decomposing the error detection process into specialized agents, each focusing on a specific aspect of multimodal understanding, will lead to significantly improved accuracy and effectiveness. We believe this approach will succeed because it mirrors the way human experts analyze mathematical errors, considering both visual and textual information in a structured manner. This project directly addresses the critical need for scalable and effective error detection in digital learning environments. By providing personalized feedback and identifying specific learning gaps, MathAgent empowers students to overcome challenges and achieve deeper understanding in mathematics.

MathAgent leverages state-of-the-art Multimodal Large Language Models (MLLMs) like InternVL2, LLaVA, and Qwen-VL for image understanding and text processing. We utilize specialized models like StructEqTable for table extraction and vit-gpt2 for image captioning. Development requires access to computational resources for training and deployment, including GPUs and cloud computing platforms. Market demand validation will involve pilot studies with educational institutions and online learning platforms, gathering user feedback and assessing learning outcome improvements. Core functionalities include: 1) Multimodal input processing (images, text, equations); 2) Error step identification; 3) Error categorization (e.g., calculation error, reasoning error); 4) Personalized feedback generation. User experience will be prioritized through intuitive interfaces and clear explanations. Performance will be evaluated using metrics like error detection accuracy, error categorization accuracy, and student learning gains.

Stream 1

MathAgent’s innovative Mixture-of-Math-Agent framework represents a significant departure from traditional monolithic MLLM approaches. By decomposing the complex task of multimodal error detection into specialized agents, we achieve a more nuanced and accurate analysis of student work. This agent-based architecture allows for targeted improvements in specific areas of multimodal understanding, leading to greater overall effectiveness. The integration of specialized models like StructEqTable for table processing further enhances the system’s ability to handle diverse mathematical content. This creative combination of specialized agents and tailored models allows MathAgent to address the unique challenges of multimodal mathematical error detection in a way that surpasses existing solutions.

Scalability will be achieved through cloud-based deployment and optimized model architectures. Load balancing and distributed processing will address potential bottlenecks. Sustainability will be ensured through efficient resource utilization and the development of energy-efficient models. Long-term user engagement will be fostered through continuous improvement based on user feedback and the incorporation of new mathematical content and error types. The modular design of MathAgent allows for easy adaptation to evolving user needs and the integration of future advancements in MLLM technology, ensuring its long-term relevance and effectiveness in the ever-changing landscape of education. Furthermore, by reducing the reliance on paper-based assessments and manual grading, MathAgent contributes to environmental sustainability.

MathAgent tackles key educational challenges by offering automated, accurate feedback on math errors, enhancing personalized learning for K-12 students, especially in underserved areas. It promotes equity and inclusion by making advanced AI accessible to diverse learners. To measure its social impact, we will track improvements in student performance, engagement, and satisfaction, as well as reductions in teacher workload and enhancements in professional development. Ensuring equity, we will assess the distribution of benefits across different demographics and locations. To stay responsive, MathAgent incorporates continuous user feedback, collaborates with educational experts, maintains an adaptive framework, and engages with the community through workshops and conferences. These strategies aim to create a positive, equitable, and inclusive impact on education.

Personal Information Collection Statement (PICS):
1. The personal data collected in this form will be used for activity-organizing, record keeping and reporting only. The collected personal data will be purged within 6 years after the event.
2. Please note that it is obligatory to provide the personal data required.
3. Your personal data collected will be kept by the LTTC and will not be transferred to outside parties.
4. You have the right to request access to and correction of information held by us about you. If you wish to access or correct your personal data, please contact our staff at lttc@eduhk.hk.
5. The University’s Privacy Policy Statement can be access at https://www.eduhk.hk/en/privacy-policy.

• I have read and agree to the competition rules and privacy policy.