Robot Pose Estimation

Solving the robot pose is a fundamental requirement for vision-based robot control, and is a process that takes considerable effort and care to make accurate. Traditional approaches require modification of the robot via markers, and subsequent deep learning approaches enabled markerless feature extraction. In this project, we utilize advanced computer vision and robot state estimation techniques for estimating the robot's pose in dynamic environments. We focus on tracking the pose of various robots, including robot manipulators, surgical robots, and snake robots.

Students & Collaborators

  • Jingpei Lu
  • Shan Lin
  • Florian Richter
  • Lucas Liang
  • Tristin Xie

Publications

Pose Estimation for Robot Manipulators via Keypoint Optimization and Sim-to-Real Transfer Jingpei Lu, Florian Richter, Michael C. Yip IEEE Robotics and Automation Letters (RA-L), vol. 7, no. 2, pp. 4622-4629, April 2022 [arXiv] [website]  


Image-based Pose Estimation and Shape Reconstruction for Robot Manipulators and Soft, Continuum Robots via Differentiable Rendering Jingpei Lu*, Fei Liu*, Cedric Girerd, Michael C. Yip (* Equal contributions) IEEE Conference on Robotics and Automation (ICRA), 2023 [arXiv]  


Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfer Jingpei Lu, Florian Richter, Michael C. Yip IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023 [arXiv] [code] [website]  


Tracking Snake-Like Robots in the Wild Using Only a Single Camera Jingpei Lu, Florian Richter, Shan Lin, Michael C. Yip IEEE Conference on Robotics and Automation (ICRA), 2024 [arXiv]

Mobility Analysis of Screw-Based Locomotion and Propulsion in Various Media

IEEE International Conference on Robotics and Automation (ICRA), pp. 7317-7323 (2023).

Design and Mechanics of Cable-Driven Rolling Diaphragm Transmission for High-Transparency Robotic Motion

IEEE International Conference on Robotics and Automation (ICRA), pp. 7338-7344 (2023).

Mobility Analysis of Screw-Based Locomotion and Propulsion in Various Media

IEEE International Conference on Robotics and Automation (ICRA), pp. 7317-7323 (2023).

Crane: a 10 degree-of-freedom, tele-surgical system for dexterous manipulation within imaging bores

D Schreiber, Z Yu, H Jiang, T Henderson, G Li, J Yu, R Zhu, AM Norbash, MC Yip

Scalable tactile sensor arrays on flexible substrates with high spatiotemporal resolution enabling slip and grip for closed-loop robotics

H. Oh, G.C. Yi, M.C. Yip, S.A. Dayeh

MPC-MPNet: Model-Predictive Motion Planning Networks for Fast, Near-Optimal Planning under Kinodynamic Constraints

L. Li, Y.L. Miao, A.H. Qureshi, M.C. Yip

​Data-driven Actuator Selection for Artificial Muscle-Powered Robots

T. Henderson, Y. Zhi, A. Liu, M.C. Yip

ARCSnake: Reconfigurable Snake-Like Robot with Archimedean Screw Propulsion for Multi-Domain Mobility

F. Richter*, P. V. Gavrilov* and H. M. Lam* and A. Degani and M. C. Yip

Exobiology Extant Life Surveyor (EELS)

K Carpenter, A Thoesen, D Mick, J Martia, M Cable, K Mitchell, S. Hovsepian, J. Jasper, N. Georgiev, R. Thakker, A. Kourchians, B. Wilcox, M. Yip, and H. Marvi.

ARCSnake: An Archimedes' Screw-Propelled, Reconfigurable Robot Snake for Complex Environments

D.A. Schreiber*, F. Richter*, A. Bilan**, P.V. Gavrilov**, H. M. Lam**, C.H. Price**, K.C. Carpenter, M.C. Yip;