Robot Pose Estimation

Solving the robot pose is a fundamental requirement for vision-based robot control, and is a process that takes considerable effort and care to make accurate. Traditional approaches require modification of the robot via markers, and subsequent deep learning approaches enabled markerless feature extraction. In this project, we utilize advanced computer vision and robot state estimation techniques for estimating the robot's pose in dynamic environments. We focus on tracking the pose of various robots, including robot manipulators, surgical robots, and snake robots.

Students & Collaborators

  • Jingpei Lu
  • Shan Lin
  • Florian Richter
  • Lucas Liang
  • Tristin Xie

Publications

Pose Estimation for Robot Manipulators via Keypoint Optimization and Sim-to-Real Transfer Jingpei Lu, Florian Richter, Michael C. Yip IEEE Robotics and Automation Letters (RA-L), vol. 7, no. 2, pp. 4622-4629, April 2022 [arXiv] [website]  


Image-based Pose Estimation and Shape Reconstruction for Robot Manipulators and Soft, Continuum Robots via Differentiable Rendering Jingpei Lu*, Fei Liu*, Cedric Girerd, Michael C. Yip (* Equal contributions) IEEE Conference on Robotics and Automation (ICRA), 2023 [arXiv]  


Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfer Jingpei Lu, Florian Richter, Michael C. Yip IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023 [arXiv] [code] [website]  


Tracking Snake-Like Robots in the Wild Using Only a Single Camera Jingpei Lu, Florian Richter, Shan Lin, Michael C. Yip IEEE Conference on Robotics and Automation (ICRA), 2024 [arXiv]

An Instrumented Glove for Improving Spasticity Assessment

S. Padmaja, P. Jonnalagedda, F. Deng, K. Douglas, L. Chukoskie, M. Yip, T.N. Ng, T. Nguyen, A. Skalsky, H. Garudadri

Designing Muscle-Powered Robotics using Super-coiled Polymers

J. Zhang and M. Yip

WoodenHaptics: A Starting Kit for Crafting Force-Reflecting Spatial Haptic Devices

J. Forsslund, M. Yip and E-L. Sallnas

High-Performance Robotic Muscles from Conductive Nylon Sewing Thread | Best paper award finalist

M. Yip and G. Niemeyer

Intra-operative Laryngoscopic Instrument for Characterizing Vocal Fold Viscoelasticity

M. Ottensmeyer, M. Yip, C. Walsh, J. Kobler, J. Heaton, and S. Zeitels