DEEP REINFORCEMENT LEARNING-BASED MAPLESS NAVIGATION ALGORITHM FOR HUSKY ROBOT WITH ADAPTIVE CURRICULUM LEARNING

Abstract

Issues. Mobile wheeled robots are widely used today in internal logistics environments of various industries, in particular in warehouses, production shops and sorting centers. However, in such environments, the robot operates under conditions of high variability, where the configuration of aisles, the placement of loads and the presence of people can change in real time. Therefore, the development of autonomous navigation of the robot in complex and dynamic environments is becoming one of the key tasks of robotics.

Research objective. The method of this work increases the efficiency of navigation of a four-wheeled autonomous mobile robot without maps (mapless) in dynamic logistics environments by applying adaptive machine learning with reinforcement (Reinforcement Learning, RL) based on the curriculum.

Methods. To achieve the set goal and solve the tasks of autonomous navigation in logistics environments, a comprehensive methodological approach was used, based on the following methods:

- simulation method: Using the PyBullet physical engine to create a virtual logistics environment. This allowed for safe and high-repetition training, simulating the real-world kinematics of the Husky four-wheeled robot;

- deep Reinforcement Learning Methods: Application and comparison of three state-of-the-art architectures: SAC (Soft Actor-Critic), PPO (Proximal Policy Optimization), TD3 (Twin Delayed DDPG);

- adaptive Curriculum Learning (ACL): A step-by-step learning technique that automatically changes the complexity of the environment (number and speed of obstacles) depending on the current success of the agent. This avoids the degradation policy at early stages.

Results. The combination of adaptive learning (ACL) of the Husky mobile robot in complex dynamic environments and hybrid reward functions increased the navigation success rate by 35–40% compared to direct learning in complex scenarios. The conducted comparative analysis of machine learning algorithms demonstrated the highest adaptability to non-stationary conditions and the best efficiency of using experience is the Soft Actor-Critic (SAC) algorithm, which achieves success in 89% of cases of application with a minimum number of collisions.

Conclusions. A comprehensive approach to map-free navigation of the autonomous mobile robot Husky in complex dynamic environments is proposed, based on a combination of adaptive learning and a physically interpreted hybrid reward function. Experiments conducted in the PyBullet simulator confirmed that a stepwise and adaptive increase in the complexity of the environment stabilizes the learning process of modern deep reinforcement learning algorithms (PPO, SAC, and TD3).