Open Category Submission

194 – Embodied conversational agents for the development of professional communication skills: Design and evaluation of a learning environment with virtual conversation agents for Retail Assistants

This project introduces an AI-driven virtual learning environment designed to help Retail Assistants enhance their communication skills through virtual conversation agents. By offering adaptive and independent learning, the platform enables users to improve their customer interaction abilities in a flexible and personalized way.
Effective communication is a key professional competence for Retail Assistants, yet traditional role-play training is often resource-intensive, requiring infrastructure, trainers, and actors. This project leverages generative AI (genKI) and augmented reality (XR) to create cost-effective, scalable, and immersive simulations that support professional skill development.
The learning environment consists of a realistic customer interaction simulation and an AI-based reflection and feedback system. The focus is on handling demanding customer conversations, a crucial skill in vocational training. Using a design-based research approach, the prototype was tested in autumn 2024 with 20 students and professionals. The initial results are promising, with participants finding the tool helpful and advocating for its integration into vocational training. The prototype serves as a foundation for broader implementation, and in partnership with one of Switzerland’s largest educational publishers, the system is being further developed and commercialized in summer 2025 to support widespread adoption in vocational education.

Situational and target group-appropriate communication is a central professional (and interdisciplinary) goal in the training of Retail Assistants. Conversation simulations and role plays are particularly suitable for developing communication skills. However, conventional simulations are resource-intensive due to their need for infrastructure, actors and experts. Simulations based on the cutting-edge technologies of generative artificial intelligence (genKI) and augmented reality (XR) show promising potential for promoting communication skills in a practical and cost-effective way.

-

In the project, an exemplary learning environment for professionals is being developed, created (version for desktop and XR glasses), and evaluated in close cooperation with practitioners using genKI and XR technologies while taking into account current didactic findings and guidelines. In addition to the simulation, the learning environment includes an AI-based didactic framework (reflexion and feedback system) that enables learners to work independently on the focused competencies.

The learning environment focuses on the “demanding customer conversation” (e.g., complaint and the customer is very upset), a relevant competence that is part of the competence area of the qualification procedure for Retail Assistants. The learning environment takes approximately 15 minutes and can be completed independently in the version for desktop or for XR glasses). It consists of a conversation with a dissatisfied customer (avatar) (see Video 1) and a reflection and feedback session with a coach (avatar) (see Video 2).

Using a design-based research approach, the design, creation, and evaluation of a prototype are being used to analyze central didactic and technological design features of AI/XR-based communication training in vocational education and training and to investigate its effect on the development of professional communication skills. The prototype was evaluated in autumn 2024 with 20 students and professionals (see images 1, 2 & 3 ). The results, which are currently being analyzed and will be published later this year, are promising. The 20 participants consider the learning environment to be helpful and wish for its permanent integration into the training program.

The prototype is intended to form the basis for a broad application in the vocational training of Retail Assistants by the implementation partners. In collaboration with one of the largest Swiss educational publishers, the learning environment is being further developed and commercialized this summer.

Our project pioneers the use of Embodied Conversational Agents in vocational education to enhance professional communication skills. By integrating AI-driven avatars with XR technology, we create a realistic and immersive learning environment where learners can engage in simulated conversations with virtual customers and receive personalized feedback from an AI-driven coach. This innovative approach enables adaptive learning and real-time feedback, which is often lacking in traditional training methods.

The design-based research methodology ensures that the learning environment evolves based on empirical findings, ensuring continuous improvement. Unlike conventional role-playing exercises, our solution allows learners to practice without fear of judgment, fostering confidence and skill acquisition. The AI-powered feedback system tailors learning experiences to individual needs, ensuring targeted skill development.

This creative fusion of AI, XR, and didactics makes the learning process engaging, effective, and scalable. By addressing the growing need for soft skills training in customer interactions, our project revolutionizes vocational education and provides a practical, technology-driven solution to real-world communication challenges.

To ensure scalability, we are designing our system to be modular and cloud-based, allowing for easy integration into existing learning management systems (LMS) and vocational training programs (not only in Swtzerland). The AI-driven feedback and coaching system can be continuously expanded to include new conversation scenarios and training modules, ensuring adaptability to different industries and training needs.

We are partnering with one of the largest Swiss educational publishers to commercialize and distribute the learning environment, ensuring widespread adoption across vocational schools and training institutions. Our focus on usability and accessibility ensures that learners from diverse backgrounds can engage with the system without extensive technical expertise.

From a sustainability perspective, our solution reduces the need for physical training materials, minimizing resource consumption. The cloud-based infrastructure ensures energy-efficient deployment, and the AI-driven feedback system helps trainers optimize their teaching efforts, reducing redundancy and increasing learning efficiency.

Through continuous updates and improvements, we ensure that our solution remains relevant, adaptable, and effective in fostering long-term engagement and professional development.

Our project directly addresses the growing demand for effective communication training, particularly in customer service professions. By providing an inclusive and accessible training environment, we support individuals in developing essential communication skills, improving their career prospects, and enhancing workplace interactions.

We prioritize equity and inclusion by designing the system to accommodate diverse learning needs, including multilingual support and adaptable difficulty levels. This ensures that learners from different educational backgrounds, cultures, and skill levels can benefit equally.

To measure social impact, we will track key performance indicators (KPIs) such as user engagement, skill improvement, and learner feedback. Additionally, we will assess the project’s long-term effects on employability and workplace satisfaction through follow-up studies.

By integrating AI and XR technologies in vocational education, we contribute to reducing skill gaps, enhancing job readiness, and promoting social mobility. Our solution not only benefits individual learners but also strengthens businesses by improving customer interactions and service quality, creating wider economic and societal benefits.

• video_2.mov
• video_1.mov
• 
• 
• 

Personal Information Collection Statement (PICS):
1. The personal data collected in this form will be used for activity-organizing, record keeping and reporting only. The collected personal data will be purged within 6 years after the event.
2. Please note that it is obligatory to provide the personal data required.
3. Your personal data collected will be kept by the LTTC and will not be transferred to outside parties.
4. You have the right to request access to and correction of information held by us about you. If you wish to access or correct your personal data, please contact our staff at lttc@eduhk.hk.
5. The University’s Privacy Policy Statement can be access at https://www.eduhk.hk/en/privacy-policy.

• I have read and agree to the competition rules and privacy policy.