3rd Workshop on Representing and Manipulating Deformable Objects @ ICRA2023

... was held on 29.05.2023 9:00 GMT+01, in London ExCel ICC Capital Suite 12.

Clothes, food, cables, and body tissue are just a few examples of deformable objects (DO) involved in both everyday and specialized tasks. Although humans are able to reliably manipulate them, automating this process using robotic platforms is still unsolved. Indeed, the high number of degrees of freedom involved in DOs undermines the effectiveness of traditional modelling, planning and control methods developed for rigid object manipulation. This paves the way for exciting questions from a research and application perspective. i) How to tractably represent the state of a deformable object? ii) How to model and simulate its highly complex and non-linear dynamics? iii) What hardware tools and platforms are best suited for grasping and manipulating? We aim to discuss these and more challenges that arise from handling deformable objects by connecting scientists from different subfields of robotics, including perception, simulation, control, and mechanics. Following the previous editions of the workshop at ICRA 2021 and ICRA 2022, the objective for the proposed third edition is to further identify promising research directions and analyze current state-of-the-art solutions with an emphasis on highlighting recent results since the 2022 workshop. We plan to facilitate this through invited talks and will foster new collaborations to connect young researchers with senior ones.

The workshop was held in hybrid mode

Links:

Content

Topics

The workshop aims to explore different aspects that will potentially allow robots to autonomously manipulate deformable objects in the near future. Enabling such manipulation is crucial for a variety of domains and tasks, e.g., domestic, industrial and surgical contexts, which involve various forms of object deformability. However, the complexity of representing and modeling the dynamics of these objects results in the lack of a current unified solution that can be adapted to a wide range of objects. Specifically, the workshop will focus on, but is not limited to, the following topics for deformable object manipulation:

  • Representation and state estimation
  • Simulation and modeling
  • Transfer from simulation to reality
  • Learning to manipulate using data-driven methods such as reinforcement learning and learning from demonstrations
  • Perception: state tracking, parameter identification, property detection (e.g. landmarks for garments) and classification, etc.
  • Control, visual servoing and planning
  • Specialized tools, e.g. grippers, and sensors
  • Multi-arm manipulation
  • Application-specific challenges: cloth folding, surgical tasks, precision agriculture, etc.

Schedule

Time Zone: GMT +01

Time Activity
09:00 - 09:15 Workshop opening
09:15 - 09:45 Yashraj Narang: Using and building simulation models for deformable-object manipulation
09:45 - 10:15 Jia Pan: Nonprehensile manipulation of cloth pieces
10:15 - 10:45 Spotlight talks #1
  • Mingrui Yu, Kangchen Lv, Changhao Wang, Masayoshi TOMIZUKA, Xiang Li - A Coarse-to-Fine Framework for Dual-Arm Manipulation of Deformable Linear Objects with Whole-Body Obstacle Avoidance [PDF] [Video]
  • Melvin Laux, Chandandeep Singh, Alexander Fabisch - Grasping 3D Deformable Objects via Reinforcement Learning: A Benchmark and Evaluation [PDF] [Video]
  • Tran Nguyen-Le, Fares Abu-Dakka, Ville Kyrki - SPONGE: Sequence Planning with Deformable-ON-Rigid Contact Prediction from Geometric Features [PDF] [Video]
  • Lawrence Yunliang Chen, Baiyu Shi, Roy Lin, Daniel Seita, Ayah Ahmad, Richard Cheng, Thomas Kollar, David Held, Ken Goldberg - Bagging by Learning to Singulate Layers Using Interactive Perception [PDF] [Video]
  • Ignacio Cuiral-Zueco, Gonzalo Lopez-Nicolas - Mesh estimation for abrupt deformations of texture-less objects [PDF] [Video]
  • Huo Shengzeng, Jihong Zhu, Hesheng Wang, David Navarro-Alarcon - A Contrastive Learning-based Planning and Control Framework for Symbolic Manipulation of Deformable Linear Objects [PDF] [Video]
  • Niklas Fiedler, Ge Gao, Fangwei Zhong, Jianwei Zhang - MultimodalClothes: A Multimodal Real-World Dataset for Robotic Clothes Classification [PDF] [Video]
  • Alex LaGrassa, Moonyoung Lee, Oliver Kroemer - Active Learning of Model Preconditions for Model-Deviation Aware Planning in Deformable Object Manipulation [PDF] [Video]
  • Yifei Dong, Florian T. Pokorny - Soft Fixtures: Towards Practical Caging-Based Manipulation of Rigid and Deformable Objects [PDF] [Video]
  • Kangchen Lv, Mingrui Yu, Yifan Pu, Xin Jiang, Gao Huang, Xiang Li - Learning to Estimate 3-D States of Deformable Linear Objects from Single-Frame Occluded Point Clouds [PDF] [Video]
  • Robert Lee, Jad Abou-Chakra, Fangyi Zhang, Peter Corke - Learning Fabric Manipulation in the Real World with Human Videos [PDF] [Video]
  • Yaru Niu, Shiyu Jin, Zeqing Zhang, Jiacheng Zhu, Ding Zhao, Liangjun Zhang - GOATS: Goal Sampling Adaptation for Scooping with Curriculum Reinforcement Learning [PDF] [Video]
10:45 - 11:30 Coffee break + Posters session #1
  • A Coarse-to-Fine Framework for Dual-Arm Manipulation of Deformable Linear Objects with Whole-Body Obstacle Avoidance
  • Grasping 3D Deformable Objects via Reinforcement Learning: A Benchmark and Evaluation
  • SPONGE: Sequence Planning with Deformable-ON-Rigid Contact Prediction from Geometric Features
  • Bagging by Learning to Singulate Layers Using Interactive Perception
  • A Coarse-to-Fine Framework for Dual-Arm Manipulation of Deformable Linear Objects with Whole-Body Obstacle Avoidance
  • Mesh estimation for abrupt deformations of texture-less objects
  • A Contrastive Learning-based Planning and Control Framework for Symbolic Manipulation of Deformable Linear Objects
  • MultimodalClothes: A Multimodal Real-World Dataset for Robotic Clothes Classification
  • Active Learning of Model Preconditions for Model-Deviation Aware Planning in Deformable Object Manipulation
  • Soft Fixtures: Towards Practical Caging-Based Manipulation of Rigid and Deformable Objects
  • Learning to Estimate 3-D States of Deformable Linear Objects from Single-Frame Occluded Point Clouds
11:30 – 12:00 Yiannis Demiris: Robot-assisted Dressing: bimanual policies for deformable object manipulation
12:00 – 12:30 Elena De Momi: AI-based techniques for improving tool-tissue interaction in robotic surgery
12:30 - 13:45 Lunch
13:45 – 14:15 Jihong Zhu: Deformable Object Manipulation: Fundamental challenges and promising applications
14:15 – 14:45 Emo Todorov: New soft-body modeling capabilities in MuJoCo
14:45 – 15:15 Carme Torras: Cloth representation and manipulation within the CLOTHILDE project
15:15 – 15:45 Spotlight talks #2
  • Xiangyu Chu, Shengzhi Wang, Minjian Feng, Jiaxi Zheng, Yuxuan Zhao, Jing Huang, K. W. Samuel Au - Model-Free Large-Scale Cloth Spreading With Mobile Manipulation: Initial Feasibility Study [PDF] [Video]
  • David Blanco-Mulero, Gokhan Alcan, Fares Abu-Dakka, Ville Kyrki - QDP: Learning to Sequentially Optimise Quasi-Static and Dynamic Manipulation Primitives for Robotic Cloth Manipulation [PDF] [Video]
  • Alberta Longhini, Marco Moletta, Alfredo Reichlin, Michael Welle, David Held, Zackory Erickson, Danica Kragic - EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamics [PDF] [Video]
  • Michał Bidziński, Piotr Kicki, Krzysztof Walas - Towards learning quasi-static models of markerless deformable linear objects for bimanual robotic manipulation [PDF] [Video]
  • Rita Laezza, Mohammadreza Shetab-Bushehri, Erol Özgür, Youcef Mezouar, Yiannis Karayiannidis - Offline Reinforcement Learning for Shape Control of Deformable Linear Objects from Limited Real Data [PDF] [Video]
  • Jingyi Xiang, Holly Dinkel - Simultaneous Shape Tracking of Multiple Deformable Linear Objects with Global-Local Topology Preservation [PDF] [Video]
  • Ananya Bal, Ashutosh Gupta, Cecilia Morales, Artur Dubrawski, John Galeotti, Howie Choset - 3D Deformation Simulation for Vascular Tissue with 2D Medical Imaging [PDF] [Video]
  • Adrien Koessler, C. Bouzgarrou - Semantic description of methods for robotic deformable object manipulation [PDF] [Video]
  • Kejia Chen, Zhenshan Bing, Fan Wu, Yuan Meng, Sami Haddadin, Alois Knoll - Contact-aware Shaping and Maintenance of Deformable Linear Objects With Fixtures [PDF] [Video]
  • Yixuan Wang, Yunzhu Li, Katherine Driggs-Campbell, Li Fei-Fei, Jiajun Wu - Dynamic-Resolution Model Learning for Object Pile Manipulation [PDF] [Video]
  • Shangjie Xue, Shuo Cheng, Pujith Kachana, Danfei Xu - Deep Field Dynamics Model for Granular Object Piles Manipulation [PDF] [Video]
15:45- 16:30 Coffee break + Posters session #2
  • Model-Free Large-Scale Cloth Spreading With Mobile Manipulation: Initial Feasibility Study
  • QDP: Learning to Sequentially Optimise Quasi-Static and Dynamic Manipulation Primitives for Robotic Cloth Manipulation
  • EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamics
  • Towards learning quasi-static models of markerless deformable linear objects for bimanual robotic manipulation
  • Offline Reinforcement Learning for Shape Control of Deformable Linear Objects from Limited Real Data
  • Simultaneous Shape Tracking of Multiple Deformable Linear Objects with Global-Local Topology Preservation
  • 3D Deformation Simulation for Vascular Tissue with 2D Medical Imaging
  • Semantic description of methods for robotic deformable object manipulation
16:30 – 17:00 Yunzhu Li: Structured Model Learning for Deformable Object Manipulation
17:00 – 17:30 Nima Fazeli: Recent advances in Multimodal Implicit Representations of Deformable Objects
17:30 – 18:00 Panel discussion
18:00 – 18:15 Closing remarks

Call for papers

We invite participants to submit extended abstracts 3+n pages, with n pages (no page-limit) for the bibliography, in the IEEE conference style.

Submissions will be reviewed by experts of their respective field. The accepted abstracts will be made available on the workshop website but will not appear in the official IEEE conference proceedings. Participants are encouraged to submit their recent work on the topics of interest mentioned above. Contributions are encouraged, but are not required, to be original.

The review process will be single-blind, meaning the submitted paper does not need to be anonymized.

Abstracts can be submitted through Microsoft CMT: https://cmt3.research.microsoft.com/WDOICRA2023.

Important dates (final extension):

  • Submission Deadline: 10.04.2023 19.04.2023 (23:59 PST)
  • Notification date: 30.04.2023 07.05.2023 (23:59 PST)
  • Final submission: 14.05.2023 21.05.2023 (23:59 PST)
  • Workshop date: 29.05.2023

IEEE RAS Computer & Robot Vision workshop award

We are happy to announce the WDO Best Paper Award sponsored by the IEEE RAS Technical Committee Computer & Robot Vision. The selected paper will receive a prize of 300$. Any contribution submitted to the workshop will be automatically considered for the award.

Update June 02, 2023: we are excited to announce that the winner of the best paper award at the workshop is:

Mingrui Yu, Kangchen Lv, Changhao Wang, Masayoshi Tomizuka, Xiang Li. A Coarse-to-Fine Framework for Dual-Arm Manipulation of Deformable Linear Objects with Whole-Body Obstacle Avoidance.

Invited Speakers (alphabetical order)

Yiannis Demiris

Yiannis Demiris


Professor
Imperial College London, UK
Personal website

Talk title: Robot-assisted Dressing: bimanual policies for deformable object manipulation

Elena De Momi

Elena De Momi


Professor
Politecnico di Milano, Italy
Personal website

Talk title: AI-based techniques for improving tool-tissue interaction in robotic surgery

Nima Fazeli

Nima Fazeli


Assistant Professor
University of Michigan, USA
Personal website

Talk title: Recent advances in Multimodal Implicit Representations of Deformable Objects

Yunzhu Li

Yunzhu Li


Postdoctoral Researcher
Stanford, USA
(Incoming) Assistant Professor, University of Illinois Urbana-Champaign, USA
Personal website

Talk title: Structured Model Learning for Deformable Object Manipulation

Yashraj Narang

Yashraj Narang


Senior Research Scientist
NVIDIA, USA
Personal website

Talk title: Using and building simulation models for deformable-object manipulation

Jia Pan

Jia Pan


Assistant Professor
University of Hong Kong, China
Personal website

Talk title: Nonprehensile manipulation of cloth pieces

Emo Todorov

Emo Todorov


Affiliate Professor, University of Washington
Founder, Roboti LLC
Personal website

Talk title: New soft-body modeling capabilities in MuJoCo

Carme Torras

Carme Torras


Professor
Institut de Robòtica i Informàtica Industrial, Spain
Personal website

Talk title: Cloth representation and manipulation within the CLOTHILDE project

Jihong Zhu

Jihong Zhu


Assistant Professor
University of York, UK
Personal website

Talk title: Deformable Object Manipulation: Fundamental challenges and promising applications

Organizers

  • Daniel Seita, Carnegie Mellon University, USA
  • Martina Lippi, Roma Tre University, Italy
  • Michael C. Welle, KTH Royal Institute of Technology, Sweden
  • Fangyi Zhang, Queensland University of Technology (QUT), Australia

Co-Organizers

  • Hang Yin, University of Copenhagen, Denmark
  • Danica Kragic, KTH Royal Institute of Technology, Sweden
  • Alessandro Marino, University of Cassino and Southern Lazio, Italy
  • David Held, Carnegie Mellon University, USA
  • Peter Corke, Queensland University of Technology (QUT), Australia

Contact

If you have any questions please contact Daniel Seita at the email: dseita AT andrew DOT cmu DOT edu