Robot Pose Estimation

Solving the robot pose is a fundamental requirement for vision-based robot control, and is a process that takes considerable effort and care to make accurate. Traditional approaches require modification of the robot via markers, and subsequent deep learning approaches enabled markerless feature extraction. In this project, we utilize advanced computer vision and robot state estimation techniques for estimating the robot's pose in dynamic environments. We focus on tracking the pose of various robots, including robot manipulators, surgical robots, and snake robots.

Students & Collaborators

  • Jingpei Lu
  • Shan Lin
  • Florian Richter
  • Lucas Liang
  • Tristin Xie

Publications

Pose Estimation for Robot Manipulators via Keypoint Optimization and Sim-to-Real Transfer Jingpei Lu, Florian Richter, Michael C. Yip IEEE Robotics and Automation Letters (RA-L), vol. 7, no. 2, pp. 4622-4629, April 2022 [arXiv] [website]  


Image-based Pose Estimation and Shape Reconstruction for Robot Manipulators and Soft, Continuum Robots via Differentiable Rendering Jingpei Lu*, Fei Liu*, Cedric Girerd, Michael C. Yip (* Equal contributions) IEEE Conference on Robotics and Automation (ICRA), 2023 [arXiv]  


Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfer Jingpei Lu, Florian Richter, Michael C. Yip IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023 [arXiv] [code] [website]  


Tracking Snake-Like Robots in the Wild Using Only a Single Camera Jingpei Lu, Florian Richter, Shan Lin, Michael C. Yip IEEE Conference on Robotics and Automation (ICRA), 2024 [arXiv]

Vibration-based Multi-Axis Force Sensing: Design, Characterization, and Modeling

W. Kuang, M.C. Yip, J. Zhang

Augmented Reality Predictive Displays to Help Mitigate the Effects of Delayed Telesurgery

F. Richter, Y.F. Zhang, Y.H. Zhi, R.K. Orosco, M.C. Yip

Variable-Stiffness Robot Orthosis enables Dynamic Activities in Limb-salvage Veterans

T. West, A. Lucas, R. Nayak, A. Liang, J. Collins, R. Miltenberger, T. Kingsbury, M. Wyatt, M. Yip

Bundled Super-Coiled Polymer Artificial Muscles: Design, Characterization, and Modeling

A. Simeonov, T. Henderson, Z. Lan, G. Sundar, A. Factor, J. Zhang and M. C. Yip

MRI guided hyper-redundant biopsy robot

D. Schreiber, A. Norbash, M. Yip

Robot Control of Endoscopic Instruments using Flexible Polymer Sheath

Aaron Gunn, Mrinal Verghese, Wesly Wong, Phil Weissbrod, M. Yip

Screw-propelled endoscopic robot

Kevin Cheng, Andrew Saad, Dmitrii Votintcev, Elaine Tanaka, Michael Yip

Three-dimensional hysteresis modeling of robotic artificial muscles with application to shape memory alloy actuators

J. Zhang and M. C. Yip

Modeling and Inverse Compensation of Hysteresis in Supercoiled Polymer Artificial Muscles

J. Zhang, K. Iyer, A. Simeonov and M. C. Yip

Three-Dimensional Hysteresis Modeling of Robotic Artificial Muscles with Application to Shape Memory Alloy Actuators

J. Zhang, M. Yip