Higher Education Category Submission

535 – LLM-enabled chatbot of supporting teachers' AI competency development based on AI-CFT

This project aims to explore innovative solutions for leveraging Large Language Models (LLMs) to enhance teachers' professional development (PD) for AI competency.
The design approach of the project is: Grounded in teachers’ AI competency framework (AI CFT) report by UNESCO (2024), this project developed LLM-driven prototype chatbots supporting teachers’ reflection on AI competency across pedagogical iteration circles, conducted pilot studies through AI CFT intervention workshops, assessed changes in teachers’ AI competencies, and based on above, is formulating an innovation and comprehensive approach to enhance teachers’ AI competency with the payable and extensible AI technical tool.

Key innovations include:
1. Calls on LLMs as a technology pedestal and adopt AI CFT as a guideline to provide a dynamic, interactive path to AI competency development rather than traditional one-time training.
2. A pedagogic cycle that integrates and supports AI competency iterations allows teachers to reflect on and improve AI competency in real teaching scenarios, and optimise through the chatbot’s feedback.
3. A data-driven AI assessment and reflection scheme is designed, and a pre-and post-testing system is incorporated in the development and design of prototypes to track the growth trajectory of teachers' AI competence.

Technical principles include:
1. LLM-driven Intelligent Interaction: Expert LLM-enabled chatbots with AI CFT knowledge base are trained through Prompt Engineering and RAG to dynamically interact with teachers.
2. Data-driven teacher AI competency assessment: Combine pre- and post-testing systems and NLP parsing technology to target teachers' AI competency development.
3. Multi-modal Integration: Provide a comprehensive solution that supports synchronous & asynchronous PD paths. Combine AI-enabled teaching loops and integrate PD paradigms.

The potential impacts include:
1. LLM-driven prototype solution for TPD on AI CFT: The project has conducted several rounds of updated iterations of the prototype chatbots based on empirical pilots. Current pilot data all respond to the positive impact of the chatbots on teachers' AI competency.
2. Increased AI competency among teachers. Teachers who participate in the pilot sessions have demonstrated growth in competency across AI foundations and applications, AI pedagogy, ethics of AI, human-centered mindset, and AI professional development. This project will allow teachers to be more creative and critical thinkers in the process of teaching with AI, avoiding technology misuse or dependency.
3. Delivering a Scalable AI TPD Paradigm. Provide policymakers with technical tools and data support to help build a replicable AI teacher training model. Facilitate the global rollout of AI CFT.

The penetration of AI in education has transformed the traditional teacher-student relationship into a teacher-AI-student dynamic. Teachers are the primary users of AI in education. They are expected to be the designers and facilitators of students’ learning with AI, the guardians of safe and ethical practice across AI-rich educational environments. The PD of teachers' AI competency cannot be overemphasized to exploit the potential benefits of AI while mitigating its risks in education settings and wider society. Addressing this global demand for AI PD, UNESCO released the AI CFT, which provides a comprehensive guideline for AI competency development.
However, despite these expectations, teachers face a significant AI competency gap. Studies indicate that the majority of educators feel underprepared to integrate AI into teaching due to a lack of structured guidance, reflection mechanisms, and professional development (PD) opportunities (UNESCO, 2023). Besides, AI-CFT’s adoption in real-world educational settings remains limited. Existing AI PD programs often follow traditional, one-time training formats that fail to support ongoing competency development. Furthermore, LLM-based solutions for teacher AI competency remain underexplored, despite the potential of generative AI to deliver personalized, interactive, and reflective professional development experiences.
Hypothesis and Proposed Solution
This project is founded on the hypothesis that:
LLM-driven, reflection-based teacher PD solutions—grounded in AI CFT and pedagogical iteration cycles—can effectively enhance teachers’ AI competency in a scalable and ethical manner.
This project is poised for success due to the synergistic integration of LLM-technical innovation and AI CFT,
Firstly, LLM-driven educational solutions have already demonstrated transformative potential, such as the real-world success of AI copilots. Besides, AI CFT provides a robust, research-backed framework involving sustainable pedagogical iteration cycles.
In such a manner, this project offers a concrete, innovative, and scalable approach to enhancing AI competency development for teachers worldwide.

N/A

Functional Architecture & Technical Workflow：
The system follows a modular LLM-driven teacher PD architecture, integrating LLMs, RAG, AI CFT knowledge base, pre/post-testing, and an interactive pedagogical cycle:
User Layer: Teachers interact with an LLM-driven chatbot via a web-based interface.
Processing Layer: AI models process inputs using Prompt Engineering & Retrieval-Augmented Generation (RAG) to personalize responses.
Data Layer: Pre/post-testing mechanisms and NLP-based analytics track teachers’ AI competency growth.
Workflow: User Query →LLM Processing (context retrieval + pedagogical logic) → Personalized Feedback & AI CFT-based Reflection → Data Logging & Assessment Tracking

Implementation of Innovative Features:
The features, technical applications and related progress are specified by the following flow.
1.LLM-powered chatbot→OpenAI GPT API, fine-tuned prompts for teacher AI competency support→Core interaction model deployed
2.Retrieval-Augmented Generation (RAG)→LangChain + AI CFT knowledge base retrieval for enhanced contextual responses→Integration in progress
3.Teacher AI competency tracking→Pre/post-testing system + NLP parsing to analyze user responses→Pilot evaluation completed
4.Pedagogical iteration & feedback loops→AI-assisted lesson reflection cycles embedded into chatbot workflow→Testing phase
5.Web-based interaction interface→Gradio-based web UI + Hugging Face deployment→Public prototype available
6.Multimodal AI competency support→Integrating synchronous & asynchronous PD paradigms→Prototype under development

Development Timeline & Implementation:
Phase 1. Research & Conceptualization: AI CFT framework integration & design of chatbot architecture (done)
Phase 2. Prototype Development: LLM fine-tuning, Prompt Engineering, and chatbot testing (done)
Phase 3. Pilot Implementation: Conduct empirical studies, collect user data (progressing)
Phase 4. Iterative Refinement: Improve based on pilot feedback, optimize system (progressing)
Phase 5. Scale-up & Deployment Final version launch & accessibility improvements (planning)

Evaluation Metrics:
The UNESCO AI CFT framework sets learning objectives, such as:
LO5.1.2: "Exemplify the new knowledge, skills, and values required by the teaching profession in local contexts in the AI era and assess the gap between their own knowledge and experience on AI and the required AI competencies."
Using this as a benchmark, our project developed measurable dimensions through literature research and expert consultation.
Teacher AI Competency Improvement: with pre/post-surveys, teacher self-assessments
Engagement & Interaction with AI Tool: with Usage data, interaction logs, NLP-based pattern analysis
Pedagogical Effectiveness: with teacher lesson reflections, focus groups
Scalability & Accessibility: with diverse educational contexts, teacher demographics

TPD models often struggle with scalability, personalization, and real-time feedback, especially under the needs for developing AI competency. Our project introduces an LLM-driven, reflection-based approach that leverages AI not just as a tool, but as an active mentor in teachers' professional growth.
Key Innovations:
1.AI-Powered Reflective Learning Cycles: Unlike static PD programs, our LLM-driven model actively engages teachers in iterative lesson reflection and refinement. This adaptive approach aligns with AI CFT learning objectives, ensuring teachers assess and enhance their AI competencies in real-world contexts.
2.RAG for Personalized Guidance: By integrating AI CFT benchmarks into a RAG-powered knowledge base, the system contextualizes teacher queries, providing targeted insights tailored to their needs. This enables dynamic, data-driven feedback rather than generic AI-generated suggestions.
3.Multi-Dimensional AI Competency Tracking: We employ a hybrid assessment model, combining self-reflection, NLP-driven analysis, and empirical data (pre/post-testing, usage logs, focus groups) to quantify AI competency growth. This innovative evaluation method ensures the solution is both scalable and measurable across diverse educational contexts.

How This Enhances Effectiveness:
This project works across various teaching contexts without requiring extensive human facilitation, focuses on empowering teachers rather than replacing them, ensuring AI adoption aligns with educational values. In this way, teachers receive adaptive, evidence-based feedback, making PD more interactive, reflective, and outcome-driven. Besides, by integrating AI pedagogy, iterative feedback loops, and competency tracking, our solution redefines TPD—offering an effective, ethical, and scalable model for the AI era.

To ensure scalability, our project employs a modular, cloud-based LLM architecture that can efficiently handle increasing user demand while remaining cost-effective and adaptable. Sustainability is embedded in both the technical infrastructure and the pedagogical design, fostering long-term engagement and continuous AI competency development for teachers.

Scalability Strategies:
Cloud-Based Deployment & API Optimization: Leveraging OpenAI API with efficient prompt engineering and RAG minimizes computational overhead while ensuring high-quality responses. Modular architecture allows parallel processing and load balancing, addressing potential bottlenecks as user numbers grow. Besides, platforms like Huggingface and GitHub provide tools for the deployment of prototypes, construction of datasets, and repository maintenance.
Adaptive AI Model & Personalization: User interaction data continuously refines chatbot responses, enabling adaptive learning pathways. Integration with teacher AI competency and background into prompts ensures personalized PD experiences, increasing engagement and reducing redundancy.
Integration with Existing PD Infrastructures: Designed to be interoperable with existing TPD systems, reducing barriers to adoption. AI-generated insights can be exported for institutional reporting and policymaker decision-making.

Adaptability to Future AI Developments:
Flexible architecture allows easy integration of future AI models and expanding knowledge bases as AI in education evolves. By leveraging scalable technology, adaptive AI, and sustainable design, this project provides a future-proof, accessible, and ethical solution for empowering teachers in the AI era.

LLM-driven TPD Tools: Ensuring Accessibility, Inclusion, and Adaptability
Our project leverages LLM-powered TPD tools, refined through multiple rounds of empirical pilot iterations. These pilots have consistently demonstrated positive impacts on teachers' AI competency, highlighting the chatbot’s effectiveness in enhancing AI literacy and pedagogical integration.
The final prototype will be (1) accessible: Designed with user-friendly interfaces and multilingual support to serve a diverse range of educators; (2)inclusive: Adapted for various educational settings, including under-resourced schools and regions with limited AI expertise; (3)adaptable: Personalized learning pathways allow teachers to engage with AI at their own pace and in contextually relevant ways.

Enhancing AI Competency Among Teachers
Participating teachers in pilot sessions have demonstrated growth across multiple dimensions of AI competency, including:
AI Foundations & Applications: Understanding AI’s role in education and leveraging AI-driven tools.
AI Pedagogy: Integrating AI into teaching while maintaining a human-centered learning approach.
Ethical AI & Responsible Use: Preventing AI misuse or over-reliance while promoting critical thinking.
AI for Professional Growth: Developing lifelong learning habits to stay updated with evolving AI trends.

Delivering a Scalable AI TPD Paradigm for Systemic Change
Beyond individual teacher impact, our project contributes to structural improvements in AI-driven teacher education by: (1)providing policymakers with technical tools & data insights to develop evidence-based AI teacher training models; (2)facilitating the global adoption of AI CFT to establish standardized AI literacy benchmarks; (3)creating an adaptable, scalable, and replicable AI-driven TPD model that supports continuous teacher upskilling at local, national, and global levels.

Measuring Social Impact & Ensuring Responsiveness
To track and improve its effectiveness, the project integrates multi-dimensional impact assessments, including:
Pre- and post-surveys: Measuring teachers' growth in AI competency.
Usage analytics: Evaluating engagement with the chatbot and learning outcomes.
Focus groups & qualitative feedback: Capturing teachers' evolving needs and real-world teaching transformations.

Personal Information Collection Statement (PICS):
1. The personal data collected in this form will be used for activity-organizing, record keeping and reporting only. The collected personal data will be purged within 6 years after the event.
2. Please note that it is obligatory to provide the personal data required.
3. Your personal data collected will be kept by the LTTC and will not be transferred to outside parties.
4. You have the right to request access to and correction of information held by us about you. If you wish to access or correct your personal data, please contact our staff at lttc@eduhk.hk.
5. The University’s Privacy Policy Statement can be access at https://www.eduhk.hk/en/privacy-policy.

• I have read and agree to the competition rules and privacy policy.