Visual Task Modeling and Automated Planning for Humanoid Robots in Education of Individuals with Autism Spectrum Disorder

Key points

• In robot-assisted education, a robot must understand how to complete a task in order to effectively guide a student through it.1
• Our work focuses on Shoebox Tasks, an activity commonly used in the education of individuals with Autism Spectrum Disorder (ASD).1
• We have developed a solver that uses a vision-based Large Language Model to convert photos of a Shoebox Task into a Planning Domain Definition Language (PDDL) representation and generate action plans for solving it.1
• Research in special educational needs indicates that students with ASD can benefit substantially from robot-assisted teaching, and our system aims to strengthen these learning opportunities while also fostering interest in robotics among individuals with ASD.1

Sources (1)

• [1]Visual Task Modeling and Automated Planning for Humanoid Robots in Education of Individuals with Autism Spectrum Disorder
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 06:57 PM

  We present a novel method that enables a semi-autonomous, camera-equipped robot to recognise the task goal and plan actions to achieve it.

  In robot-assisted education, a robot must understand how to complete a task in order to effectively guide a student through it.

Extractive summary: sentences quoted from the sources.