Open Category Submission

647 – JoBot

JobBot is an innovative AI-powered training system designed to enhance job interview skills for youths with Special Education Needs (SEN). Our solution combines conversational AI (ChatGPT) with interactive robotics to create personalized, engaging interview simulations that address the unique challenges faced by neurodiverse job seekers.
Key Innovations:
1. Human-Centric AI Interaction: Our robot mimics natural movements and expressions to reduce anxiety and increase engagement
2. Adaptive Learning System: ChatGPT integration provides multilingual support (Cantonese/Putonghua/English) with responses tailored to individual learning patterns
3. Immersive Training Environment: Pre-set recruitment scenarios with customizable backgrounds simulate real interview conditions
Technical Implementation:
• Proprietary algorithm creates personalized training profiles based on user interactions
• Dialogue history analysis enables progressive difficulty adjustment
• Real-time feedback system highlights strengths and improvement areas
Proven Impact:
In our 4-week preliminary study with 7 SEN youths (21-23 years old):
• 35.71% increase in engagement
• 22.22% improvement in self-reported motivation
One participant noted: "The training helped me feel confident to handle interviews independently."
Broader Significance:
JobBot addresses critical gaps in conventional SEN job interview training by:
• Overcoming communication barriers through adaptive technology
• Providing scalable, individualized preparation
• Building confidence through repeated low-stakes practice
Our solution demonstrates how assistive technologies can create more equitable pathways to employment. By combining emotional intelligence (robot interactions) with cognitive support (AI dialogue), JobBot empowers SEN youths to showcase their abilities effectively, ultimately improving their employment prospects and fostering greater social inclusion.

• JobBot-ppt.pdf

Youths with SEN ofen encounters difficulties in conventional job interview training due to its rigid, one-size-fits-all approach. According to HK01 news, Currently, there are approximately 67,000 primary and secondary school students with Special Education Needs (SEN) in Hong Kong, accounting for 10% of all students. After graduating from school, they must "overcome numerous obstacles" just to integrate into society—facing highly competitive academic requirements, scarce job opportunities, and challenging workplace relationships... Each hurdle looms like an insurmountable mountain before them, as if the entire world believes: "SEN means inferiority, means incapability." As a result, some give up trying altogether, retreating to their homes and fading into forgotten corners.
Through our work with Youth Academy for Special Educational Needs (YaSEN), we observed three critical gaps: (1) conventional methods lack personalization for different communication styles, (2) human-led training can be cost-prohibitive for repeated practice, and (3) many programs may not address the sensory and anxiety challenges common among SEN individuals.
Our inspiration came from witnessing talented SEN individuals struggle to showcase their abilities in interview settings. One student was very nervous as he had never conducted job interview training before. After few trials, he was able to name different camera brands after few trials. This highlighted how conventional interview formats create unnecessary barriers for youth with SEN.
We hypothesized that an AI-robotic system could overcome these limitations by:
1. Providing a judgment-free environment for unlimited practice
2. Adapting to individual communication preferences in real-time
3. Reducing social anxiety through predictable robotic interactions
Three key factors suggest our approach will succeed:
First, pilot data shown that learning engagement of those youths improved by 35.71%. Second, the combination of visual (robot) and verbal (ChatGPT) interaction aligns with multimodal learning theory. Third, our multilingual capability addresses Hong Kong's specific linguistic diversity needs.
The project's educational relevance lies in its potential to:
- Interactive and personalized interview preparation through scalable technology
- Bridge the gap between school training and workplace expectations
- Provide measurable skill development through data-driven feedback
By focusing on interview skills, a critical gateway to employment, JobBot addresses both immediate vocational needs and broader societal inclusion. Our solution embraces educational equity that requires adapting systems to neurodiverse minds. The positive pilot outcomes suggest this adaptive technology approach could redefine employability training for SEN populations.

for Streams 1&2 only

JobBot’s system architecture comprises three integrated modules:
1. User Interface Layer:
o Interactive robots (Kebbi/BoonBoon) with interactive gesture, animation, sound and movement
o Web- and Tablet-based control panel for educators to adjust scenarios
2. AI Processing Layer:
o Generative AI is utilized with localization design to fit the interview with local context
o Automatic Speech Recognition (AST) and Speech-to-Text (STT) functions were utilized to conduct instant interview training.
o Real-time speech analysis for tone, stress, and communication monitoring
3. Data Analytics Layer:
o All interaction metrics will be collected (e.g., response coherence, eye contact, correctness of answer, facial expression, and posture)
o A data visualization dashboard will be developed for progress visualization
Innovative Feature Implementation
o Physiological sensors (e.g., smart watches) feed data to adjust question complexity
o Robot movements → Enhance engagement
o Generative AI → personalization → Improve response quality

JobBot empowers youths to select from multiple industry-specific interview scenarios, ensuring training adapts to their career aspirations without restricting their potential. The AI dynamically generates market-relevant questions based on real-time labour trends, equipping users with skills that match current employer demands.
- "Real-life scenarios" → "Industry-specific interview scenarios" (more precise)
- Added "without restricting their potential" (addresses SEN stigma)
- "Diverse questions" → "Market-relevant questions" (ties directly to employability)
- Specified "real-time labour trends" (shows tech capability)

JobBot is designed for scalable and sustainable impact through a cloud-based AI architecture that supports concurrent users with automatic load balancing, while our modular system easily accommodates new languages and user groups - including Non-Chinese speaking youth, mainstream students, and vocational trainees through specialized interview tracks. The solution's multi-channel deployment spans robotic kiosks in schools/NGOs, mobile tablets for community centers, and web portals for home use, ensuring universal accessibility. We maintain sustainability through paperless and carbon-neutral AWS servers, while our growing network of different SEN school partnerships and "train-the-trainer" programs ensures local relevance and long-term adoption. This multilayered approach allows JobBot to expand its impact while maintaining core efficacy standards across diverse user needs and environments.

JobBot addresses Hong Kong's pressing issue of employment inequality for youths with SEN, where only 32% secure jobs versus 89% of neurotypical peers, by providing accessible interview training that adapts to neurodiverse communication styles through AI-powered, judgment-free simulations. Our solution promotes social equity by offering multilingual support (Cantonese/English/Putonghua), affordable access via school/NGO partnerships. We ensure ongoing community responsiveness through quarterly co-design workshops with SEN users and employers, while aligning with UN SDG 8 and Hong Kong's poverty reduction goals. By tracking both immediate skill gains and long-term career progression through employer partnerships, JobBot transforms interviews from barriers into opportunities, creating an inclusive workforce that values neurodiversity as an asset.

Personal Information Collection Statement (PICS):
1. The personal data collected in this form will be used for activity-organizing, record keeping and reporting only. The collected personal data will be purged within 6 years after the event.
2. Please note that it is obligatory to provide the personal data required.
3. Your personal data collected will be kept by the LTTC and will not be transferred to outside parties.
4. You have the right to request access to and correction of information held by us about you. If you wish to access or correct your personal data, please contact our staff at lttc@eduhk.hk.
5. The University’s Privacy Policy Statement can be access at https://www.eduhk.hk/en/privacy-policy.

• I have read and agree to the competition rules and privacy policy.