2nd Workshop on Representing and Manipulating Deformable Objects @ ICRA2022 was held on 23.05.2022 8:30 GMT-04, Room 113C

Clothes, food, cables, and body tissue are just a few examples of deformable objects involved in both everyday and specialized tasks. Although humans are able to reliably manipulate them, automating this process using robotic platforms remains challenging. Indeed, the high number of degrees of freedom involved undermines the effectiveness of traditional modelling, planning and control methods developed for rigid object manipulation. This paves the way for exciting questions from a research and application perspective. i) How to tractably represent the state of a deformable object? ii) How to model and simulate its highly complex and non-linear dynamics? iii) What hardware tools and platforms are best suited for grasping and manipulating? We aim to discuss these and more challenges that arise from handling deformable objects by connecting scientists from different subfields of robotics, including perception, simulation, control, and mechanics.

The workshop was held in hybrid mode with free remote participation

Links:

Content

Topics

The workshop aims to explore different aspects that will potentially allow robots to autonomously manipulate deformable objects in the near future. Enabling such manipulation is crucial for a variety of domains and tasks, e.g., domestic, industrial and surgical contexts, which involve various forms of object deformability. However, the complexity of representing and modeling the dynamics of these objects results in the lack of a current unified solution that can be adapted to a wide range of objects. Specifically, the workshop will focus on, but is not limited to, the following topics for deformable object manipulation:

  • Representation and state estimation
  • Simulation and modeling
  • Transfer from simulation to reality
  • Learning to manipulate using data-driven methods such as reinforcement learning and learning from demonstrations
  • Perception: state tracking, parameter identification, property detection (e.g. landmarks for garments) and classification, etc.
  • Control, visual servoing and planning
  • Specialized tools, e.g. grippers, and sensors
  • Multi-arm manipulation
  • Application-specific challenges: cloth folding, surgical tasks, precision agriculture, etc.

Schedule [PDF]

Time Zone: GMT-04

Time Activity
08:30 - 08:45 Welcome [Video]
08:45 - 09:15 Kimitoshi Yamazaki: Cloth manipulation based on hierarchical shape prediction [Video]
09:15 - 09:45 Youcef Mezouar (Mohammad Al Khatib): The european project Softmanbot : handling deformable and flexible materials for the industry [Video]
09:45 - 10:25 Spotlight talks #1 [Video]
  • Mingrui Yu, Hanzhong Zhong, Xiang Li - Shape Control of Deformable Linear Objects with Offline and Online Learning of Local Linear Deformation Models [PDF] [Video]
  • Louis Dehaybe, Olivier Brüls - An SE(3)-based formulation of the shape servoing problem [PDF] [Video]
  • Peter Mitrano, Dmitry Berenson - Data Augmentation for Online Learning of Rope Manipulation [PDF] [Video]
  • Thomas Lips, Victor-Louis De Gusseme, Francis wyffels - Learning Keypoints from Synthetic Data for Robotic Cloth Folding [PDF] [Video]
  • Irene Garcia-Camacho, Júlia Borràs, Berk Calli, Adam Norton, Guillem Alenyà - Cloth manipulation and perception competition [PDF] [Video]
  • James A. Preiss, David Millard, Tao Yao, Gaurav S. Sukhatme - Deep Recurrent Models for Nonlinear Model Predictive Control in Deformable Manipulation Tasks [PDF] [Video]
10:25 - 11:00 Coffee break + Posters session #1
  • Alex LaGrassa, Oliver Kroemer - Planning with Model Preconditions for Water Manipulation [PDF] [Video]
  • Peter Mitrano, Dmitry Berenson - Data Augmentation for Online Learning of Rope Manipulation [PDF] [Video]
  • Thomas Lips, Victor-Louis De Gusseme, Francis wyffels - Learning Keypoints from Synthetic Data for Robotic Cloth Folding [PDF] [Video]
  • Irene Garcia-Camacho, Júlia Borràs, Berk Calli, Adam Norton, Guillem Alenyà - Cloth manipulation and perception competition [PDF] [Video]
  • James A. Preiss, David Millard, Tao Yao, Gaurav S. Sukhatme - Deep Recurrent Models for Nonlinear Model Predictive Control in Deformable Manipulation Tasks [PDF] [Video]
11:00 – 11:30 Ken Goldberg: Real2Sim2Real: A Model for Deep Learning to Manipulate Deformable Objects [Video]
11:30 – 12:00 Andrea Cherubini: See and shape: vision-based robot manipulation of non-rigid objects [Video]
12:00 – 12:40 Spotlight talks #2 [Video]
  • Rita Laezza, Finn Süberkrüb, Yiannis Karayiannidis - Blind Manipulation of Deformable Linear Objects Based on Force Information from Environmental Contacts [PDF] [Video]
  • Holly Dinkel, Jingyi Xiang, Harry Zhao, Brian Coltin, Trey Smith, Timothy Bretl - Wire Point Cloud Instance Segmentation from RGBD Imagery with Mask R-CNN Contacts [PDF] [Video]
  • Shengyin Wang, Matteo Leonetti, Mehmet Dogar - Goal-Conditioned Model Simplification for Deformable Object Manipulation [PDF] [Video]
  • Shaoxiong Yao, Kris Hauser - Online Estimation of Point-based Volumetric Stiffness Model Using Joint Torque Sensors [PDF] [Video]
  • Azarakhsh Keipour, Mohammadreza Mousaei, Maryam Bandari, Stefan Schaal, Sebastian Scherer - Detection and Physical Interaction with Deformable Linear Objects [PDF] [Video]
  • Cristiana de Farias, Brahim Tamadazte, Rustam Stolkin, Naresh Marturi - Grasp Transfer for Deformable Objects by Functional Map Correspondence [PDF] [Video]
12:40 - 13:45 Lunch [Video]
13:45 – 14:15 Shuran Song: The Reasonable Effectiveness of Dynamic Manipulation for Deformable Objects [Video]
14:15 – 14:45 Rika Antonova: Distributional Representations and Scalable Simulations for Real-to-Sim with Deformables [Video]
14:45 – 15:15 Spotlight talks #3 [Video]
  • Youngsun Wi, Pete Florence, Andy Zeng, Nima Fazeli - VIRDO: Visio-tactile Implicit Representations of Deformable Objects [PDF] [Video]
  • Haochen Shi, Huazhe Xu, Zhiao Huang, Yunzhu Li, Jiajun Wu - RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networks [PDF] [Video]
  • Gautam Salhotra, I-Chun Arthur Liu, Marcus Dominguez-Kuhne, Gaurav S. Sukhatme - Learning Deformable Manipulation from Expert Demonstrations [PDF] [Video]
  • Alex LaGrassa, Oliver Kroemer - Planning with Model Preconditions for Water Manipulation [PDF] [Video]
  • Priya Sundaresan, Rika Antonova, Jeannette Bohg - DiffCloud: Real-to-Sim from Point Clouds with Differentiable Simulation and Rendering of Deformable Objects [PDF] [Video]
15:15- 16:00 Coffee break + Posters session #2
  • Rita Laezza, Finn Süberkrüb, Yiannis Karayiannidis - Blind Manipulation of Deformable Linear Objects Based on Force Information from Environmental Contacts [PDF] [Video]
  • Shaoxiong Yao, Kris Hauser - Online Estimation of Point-based Volumetric Stiffness Model Using Joint Torque Sensors [PDF] [Video]
  • Youngsun Wi, Pete Florence, Andy Zeng, Nima Fazeli - VIRDO: Visio-tactile Implicit Representations of Deformable Object [PDF] [Video]
  • Gautam Salhotra, I-Chun Arthur Liu, Marcus Dominguez-Kuhne, Gaurav S. Sukhatme - Learning Deformable Manipulation from Expert Demonstrations [PDF] [Video]
  • Priya Sundaresan, Rika Antonova, Jeannette Bohg - DiffCloud: Real-to-Sim from Point Clouds with Differentiable Simulation and Rendering of Deformable Objects [PDF] [Video]
16:00 – 16:30 Alan Kuntz & Isabella Huang: Large-Scale Simulation for Calibration-Free Sim to Real Transfer of Deformable Object Manipulation [Video]
16:30 – 17:00 Carolyn Matl: Deformable Elasto-Plastic Object Shaping using an Elastic Hand [Video]
17:00 – 17:30 Panel discussion [Video]
17:30 – 17:45 Closing remarks

Call for papers

We invite participants to submit extended abstracts 3+n pages, with n pages (no page-limit) for the bibliography, in the IEEE conference style.

Submissions will be reviewed by experts of their respective field. The accepted abstracts will be made available on the workshop website but will not appear in the official IEEE conference proceedings. Participants are encouraged to submit their recent work on the topics of interest mentioned above. Contributions are encouraged, but are not required, to be original.

The review process will be single-blind, meaning the submitted paper does not need to be anonymized.

Abstracts can be submitted through Microsoft CMT: https://cmt3.research.microsoft.com/WDOICRA2022.

Important dates:

  • Submission Deadline: 05.04.2022 AOE 17.04.2022 AOE
  • Notification date: 30.04.2022 AOE 06.05.2022 AOE
  • Final submission: 14.05.2022 AOE 16.05.2022 AOE
  • Workshop date: 23.05.2022 AOE

Invited Speakers (alphabetical order)

Rika Antonova

Rika Antonova


Postdoctoral Researcher
Stanford University, United States
Personal website

Talk title: Distributional Representations and Scalable Simulations for Real-to-Sim with Deformables

Andrea Cherubini

Andrea Cherubini


Associate Professor
University of Montpellier, France
Personal website

Talk title: See and shape: vision-based robot manipulation
of non-rigid objects

Ken Goldberg

Ken Goldberg


Professor, Industrial Engineering and Operations Research
William S. Floyd Jr. Distinguished Chair in Engineering
UC Berkeley, United States
Personal website

Talk title: A Model for Deep Learning to Manipulate Deformable Objects

Isabella Huang

Isabella Huang


PhD Student
UC Berkeley, United States
Personal website

Talk title: Large-Scale Simulation for Calibration-Free Sim to Real
Transfer of Deformable Object Manipulation (joint talk with Tucker Hermans)

Alan Kuntz

Alan Kuntz


Assistant Professor
University of Utah, United States
Personal website

Talk title: Large-Scale Simulation for Calibration-Free Sim to Real
Transfer of Deformable Object Manipulation (joint talk with Isabella Huang)

Carolyn Matl

Carolyn Matl


Research Scientist at Toyota Research Institute
United States
Personal website

Talk title: Deformable Elasto-Plastic Object Shaping using an Elastic Hand

Youcef Mezouar

Youcef Mezouar


Professor at SIGMA'Clermont, Institut Pascal UMR 6602 CNRS
Université Clermont Auvergne/Sigma, France
Personal website

Talk title: The european project Softmanbot : handling deformable
and flexible materials for the industry

Shuran Song

Shuran Song


Assistant Professor
Columbia University, United States
Personal website

Talk title: The Reasonable Effectiveness of Dynamic Manipulation for Deformable Objects

Kimitoshi Yamazaki

Kimitoshi Yamazaki


Professor
Shinshu University, Japan
Personal website

Talk title: Cloth manipulation based on hierarchical shape prediction

Organizers (alphabetical order)

  • Martina Lippi, Roma Tre University, Italy
  • Daniel Seita, Carnegie Mellon University, USA
  • Michael C. Welle, KTH Royal Institute of Technology, Sweden

Co-Organizers

  • Hang Yin, KTH Royal Institute of Technology, Sweden
  • Danica Kragic, KTH Royal Institute of Technology, Sweden
  • Alessandro Marino, University of Cassino and Southern Lazio, Italy
  • David Held, Carnegie Mellon University, USA
  • Yiannis Karayiannidis, Chalmers University of Technology, Sweden

Contact

If you have any questions please contact Martina Lippi at the email: martina.lippi AT uniroma3 DOT it