Open Category Submission

430 – InstrucTwin: Your Trusted Virtual Teaching Assistant

This project aims to build InstrucTwin, a GenAI-based platform and tools to help instructors plan their lectures and lessons based on the approved course specifications (i.e., CLOs), generate course materials (e.g., lecture slides, formative assessments and summative assessments like quizzes and exams), evaluate the performance of the instructor during lecture, and provide recommendations on the instructor’s performance and students performances based on the various assessments. One of the key innovations of the proposed tool is the ability to do real-time observations of audio and video data (acquired via microphones and cameras) during lecture sessions, analyze the data, and provide recommendations in terms of actions that can either be applied during the session or in later sessions. We envision using advanced generative AI and other AI technologies, such as automatic speech recognition, natural language processing, and computer vision, uniquely to create the optimal teaching assistant.
The proposed platform will consist of multiple tools. The first tool is for lesson planning, which allows instructors to plan their lessons using chatbot-based AI interactively. The second tool is the plan execution tool, which allows instructors to track their lesson delivery and execution and also conduct formative assessments during class. This tool also analyzes the lecture activities and provides recommendations to help the instructor optimize the lecture time to maximize student learning. The final tool is for building summative assessments based on the lectures and other course activities to assess the learning outcomes of the course.
The project’s potential impact is significant. Instructors may face challenges in their teaching assessments. This platform provides tailored AI assistance to enhance learning experiences across educational settings. It benefits both students and instructors in academics and training. Future efforts will focus on refining AI models to incorporate the latest educational best practices and expanding beyond STEM fields.

As a faculty member, I am always thinking about improving my teaching to help our students acquire the knowledge, skills, and values for the course I am teaching. One area that feels lacking is improving my formative assessments, which can help me understand what concepts my students are struggling with and help with improving their engagement during class with effective feedback.
Furthermore, in some cases, instructors may have to teach multiple different courses with a large number of students, each with limited time and teaching assistants. Thus, there is a need to utilize AI technologies and leverage them to support the educational process. Given the current advances in Generative AI, it is possible to leverage them to support instructors and improve the learning experience for the students.
In this project, we propose InstrucTwin: a software platform that utilizes generative AI technologies to assist instructors with lesson planning, execution, and delivery. It can also offer recommendations and suggestions during class sessions (i.e., in real-time) or after class sessions. We hypothesize that this software platform can enhance the teaching and learning experiences.
Lesson planning involves developing lesson plans with AI to align with a specific desired teaching style while meeting the course learning outcomes. Plan execution and delivery focus on enhancing the learning experience by generating real-time interactive polls and formative assessments, along with providing personalized feedback based on individual student participation histories. The platform can also deliver recommendations and notifications in real time or compile a report of the instructor’s delivery for review after class.
The proposed AI-based software platform is expected to succeed in improving the teaching and learning experience in situations where there are limited resources available for instructors. Building and deploying software tools and platforms as teaching and learning resources can be more cost-effective than traditional human-based resources.

The solution builds a software platform with server-side and client-side components, using Google Firebase for implementation and providing web and mobile client applications. The Firebase framework is a cloud-based platform that facilitates scalable, cloud-native applications, enabling us to scale as needed. For AI technologies, we plan to use Gemini, Vertex AI, and other hosted LLMs to assist with reasoning and agentic code generation. Additionally, we will employ Speech-to-Text and Image Captioning technologies to enhance LLMs' understanding of the context within a given user session.
Required resources include cloud credits, one full-stack developer, one QA staff member, and educational consultants by field. Initially, we will focus on computer science and engineering, including Computer Science, Computer Engineering, Data Science, Artificial Intelligence, and Software Engineering. In later stages, we plan to expand to additional STEM fields.
We will market our solution through two programs: a partnership with educational institutions and an ambassador program for individual instructors. Both programs will grant access to our platform to validate demand and impact.
The core functionalities are as follows. An AI-based lesson planner. Lesson plans and all of the platform’s data will be stored in a secured database management system. An AI-based formative assessments and feedback generation module to generate different kinds of formative assessments (e.g., polls, quizzes, e.g., parsons problems). A plan execution validator and recommender module that captures multimodal data of the instructors and the students to validate lesson plan execution and provide suggestions and recommendations during and after class sessions.
We will ensure a positive user experience through intuitive design and strong customer and IT support. Performance metrics include platform latency and UI/UX indicators like task success rates and time on task. We will also utilize qualitative metrics and satisfaction surveys, including the System Usability Scale (SUS) and Customer Satisfaction Score (CSAT).

Not applicable for Stream 2.

The idea represents an innovative and creative solution that leverages advanced cloud and AI technologies to provide real-time instructional support. It introduces new approaches that transform traditional educational practices, such as enhancing student engagement through personalized learning experiences and integrating AI tools to automate and facilitate real-time pop-up questions, quizzes, and discussions. Also, it helps instructors with lesson planning for in-class activities and assessments. The creativity lies in applying AI-driven instructor assistance platform technology, providing unique solutions that resonate with instructors and learners. This innovative integration not only improves student engagement and formative assessment but also improves the instructor’s performance by providing continuous evaluation, recommendations, and suggestions for improvement, which can significantly boost the effectiveness of education.
With InstrucTwin, instructors can get personalized insights into student progress based on course learning outcomes. AI-driven analytics can identify areas where students need extra help, thereby allowing tailored educational support. AI-driven analytics can recommend instructional improvements and assist instructors in creating engaging, interactive content. As a result, InstrucTwin empowers educators to deliver more effective, responsive, and personalized teaching experiences.
Furthermore, InstrucTwin uses multimodal data to provide comprehensive planning, analysis, and recommendations. Real-time audio and video analysis of the lecture session is used to help instructors optimize their teaching and improve the student’s learning experiences.
The AI models can be continuously improved as education research progresses. InstrucTwin can recommend state-of-the-art teaching methodologies as they are published. Moreover, we plan to build additional tools to enable education research.

InstrucTwin uses cloud-based deployment to scale with increasing user demands. This includes cloud-native databases and serverless architecture. As the demand and the workload on the platform increase, more cloud resources can be allocated elastically to meet the required demand. When the workload decreases, the computational resource allocations for InstrucTwin are reduced to ensure cost-effective operations.

InstructTwin focuses on digital technologies, which reduce our reliance on physical paper. Furthermore, we plan to optimize our use of cloud resources and rely on effective AI models to reduce the negative impact on the environment by utilizing low CO2 data centers.

Moreover, We also plan to incorporate state-of-the-art research and best practices from the academic research literature. This makes the application of best practices at the fingertips of all educators.

We plan to build integration with existing LMS tools and platforms to ensure that our users can still utilize tools that they are already familiar with.

Finally, we plan to organize workshops and seminars at major educational conferences as well as online to keep our engagement with our users.

InstrucTwin addresses several social issues for both instructors and students. InstrucTwin aims to reduce inconsistencies in instructional quality, reduce instructor burnout, and improve job satisfaction globally. There is a lack of personalized education for diverse learning needs, which InstrucTwin can address. In addition, InstrucTwin can address biases and unequal student engagement, as well as insufficient emotional and psychological student support due to instructors' workload.
To evaluate the impact of instructional quality, monitor improvements in student achievement scores, and observe reductions in educational performance gaps across diverse schools. Also, using InstrucTwin can reduce heavy burdens on instructors and improve their teaching quality. The metrics for instructor burnout are increased instructor satisfaction and reduced turnover rates, which were measured by surveys that can be sent and analyzed using InstrucTwin.
Moreover, other metrics, such as enhanced instructor performance ratings derived from student feedback, as well as increased adaptability and responsiveness in teaching methods, serve as measures for personalized education catering to diversity. Furthermore, the metrics for assessing bias reduction and equality in student engagement include heightened instructor awareness and modifications informed by AI-driven student engagement analytics, alongside a greater sense of inclusion among students as indicated by periodic surveys. Finally, evaluating the increased availability of instructors for mentorship and emotional support, along with improved student well-being indicators such as stress levels, engagement, and reported satisfaction, reflects the progress made in alleviating instructors' workload through InstrucTwin support.

Personal Information Collection Statement (PICS):
1. The personal data collected in this form will be used for activity-organizing, record keeping and reporting only. The collected personal data will be purged within 6 years after the event.
2. Please note that it is obligatory to provide the personal data required.
3. Your personal data collected will be kept by the LTTC and will not be transferred to outside parties.
4. You have the right to request access to and correction of information held by us about you. If you wish to access or correct your personal data, please contact our staff at lttc@eduhk.hk.
5. The University’s Privacy Policy Statement can be access at https://www.eduhk.hk/en/privacy-policy.

• I have read and agree to the competition rules and privacy policy.