Vision-Based Safe Control for Robotic Manipulators

Abstract

This thesis presents a comprehensive framework for vision-based safe control of robotic manipulators, emphasizing image-based visual servoing (IBVS) combined with optimization-based collision avoidance, nonlinear model predictive control (NMPC), and disturbance observer (DOB) techniques. A systematic exploration of these methodologies is undertaken to enable robotic manipulators to perform precise and safe interactions within dynamic and uncertain environments. The proposed IBVS framework integrates robust visual feedback with predictive capa- bilities, addressing real-time obstacle avoidance while maintaining high control accuracy. Simulation studies using Gazebo and real-world vali- dations with NVIDIA’s Isaac ROS AprilTag demonstrate the efficacy and robustness of the control strategies under various challenging conditions, including dynamic disturbances, sensor noise, and calibration uncertain- ties. Additionally, advanced numerical solvers such as CasADi and IPOPT are employed to efficiently handle the optimization-based control tasks. This work significantly advances the practical deployment of vision-based robotic manipulation by enhancing safety, reliability, and computational ef- ficiency, paving the way for broader applications in industrial automation, autonomous navigation, and collaborative robotics.