ZePHyR: Zero-shot Pose Hypothesis Rating
Brian Okorn*
Qiao Gu*
Martial Hebert
David Held
Pose hypotheses scored using Zero-shot Pose Hypothesis Rating on novel drill object, reconstructed at test time. Poses are outlined in color corresponding to score, with highly rated poses in red transitioning to lower ones in blue.


Pose estimation is a basic module in many robot manipulation pipelines. Estimating the pose of objects in the environment can be useful for grasping, motion planning, or manipulation. However, current state-of-the-art methods for pose estimation either rely on large annotated training sets or simulated data. Further, the long training times for these methods prohibit quick interaction with novel objects. To address these issues, we introduce a novel method for zero-shot object pose estimation in clutter. Our approach uses a hypothesis generation and scoring framework, with a focus on learning a scoring function that generalizes to objects not used for training. We achieve zero-shot generalization by rating hypotheses as a function of unordered point differences. We evaluate our method on challenging datasets with both textured and untextured objects in cluttered scenes and demonstrate that our method significantly outperforms previous methods on this task. We also demonstrate how our system can be used by quickly scanning and building a model of a novel object, which can immediately be used by our method for pose estimation. Our work allows users to estimate the pose of novel objects without requiring any retraining.

Teaser Video


Our method first projects the sampled model points M onto the observation I according to a pose hypothesis hi. Then Di are extracted as the point-wise differences between the observation and the projected model points, describing the alignment of the pose hypothesis at each projected point. A network takes in Di and regresses to a score si for each pose hi which evaluates how well the pose matches the observation.

Paper and Supplementary Material

B. Okorn*, Q. Gu*, M. Hebert, D. Held.
ZePHyR: Zero-shot Pose Hypothesis Scoring.
In ICRA, 2021.
(hosted on ArXiv)



This template was originally made by Phillip Isola and Richard Zhang for a colorful ECCV project; the code can be found here.