5th Workshop: Reflections on Representations and Manipulating Deformable Objects @ ICRA2025

Over the past five years, significant progress has been made in deformable object (DO) manipulation, from breakthroughs in representation, simulation, and control to the integration of machine learning and large models for improved perception. Despite advancements, key challenges remain, particularly in developing a unified framework for DO representation across different domains. The "5th Workshop: Reflections on Representations and Manipulating Deformable Objects" aims to critically review these advancements and identify future research priorities. Topics include state representation, non-linear dynamic modeling, and the application of models like vision-language models (VLMs). The workshop will highlight advances in learning methods, such as reinforcement and imitation learning, which have improved robotic dexterity. It will also explore the role of low-cost hardware and open datasets in accelerating research.

The workshop was held in hybrid mode

Links:

Content

Topics

The "5th Workshop: Reflections on Representations and Manipulating Deformable Objects" seeks to critically examine the progress made in enabling robots to autonomously manipulate deformable objects and to outline the remaining challenges. These objects, which are present in various domains such as domestic, industrial, and surgical environments, continue to pose significant challenges for robotics due to their complex dynamics and non-linear behaviors. Despite advances in representation, simulation, and control, there is still no unified solution that can generalize across the broad range of deformable objects. In recent years, the integration of large-scale pre-trained models, including vision-language models (VLMs) and foundation models, has opened new avenues for solving language-conditioned tasks and improving sample efficiency in robotic learning. These models, combined with recent developments in imitation learning, reinforcement learning, and advanced 3D representation models, have enhanced robots' ability to perform more complex, dexterous, and long-horizon tasks. Furthermore, the release of new simulators, open datasets, and accessible low-cost robotic hardware has greatly lowered the barriers to reproducible research, benchmarking, and reuse of data, allowing researchers to focus on deeper challenges in deformable object manipulation. This workshop will explore how these technological advances can be leveraged to overcome the persistent challenges in deformable object manipulation. Key topics will include:

  • Representation and state estimation
  • Simulation and modeling
  • Transfer from simulation to reality
  • Learning to manipulate using data-driven methods such as reinforcement learning and learning from demonstrations
  • Perception: state tracking, parameter identification, property detection (e.g. landmarks for garments) and classification, etc.
  • Control, visual servoing and planning
  • Use of foundation models, such as large vision and language models, and associated large datasets
  • Specialized tools, e.g. grippers, and sensors

    • Workshop format

    The workshop will include:

    • Invited talks by selected speakers, each consisting of about 25 minutes of presentation followed by 5 minutes for Q&A;<\li>
    • Accepted extended abstracts and Reflections (3 pages with unlimited references and appendix) presented in poster sessions and selected spotlight talks. In case of a hybrid or virtual workshop, we will ask for pre-recorded spotlight talks for a smoother execution in case of connection issues. However, for each selected contribution, at least one author will be required to be present during the workshop for a live Q&A session;
    • A panel discussion at the end of the workshop, moderated by the organizers, for discussing challenges and promising directions for deformable object manipulation with experts of the field.

Schedule

Time Zone: GMT -04

Time Activity
08:30 - 08:45 Workshop opening
08:45 - 09:15 Jeannette Bohg [In person] - Object-centric or not: How to represent Deformables for Manipulation?
09:15 - 10:00 Spotlight Talks #1 (TBA)
  • TBA
10:00 - 11:00 Coffee Break + Poster Presentation
11:00 - 11:30 Yiannis Demiris [Remote] - Manipulation of Deformable Objects in Assistive Tasks
11:30 - 12:00 Shuran Song [In person] - TBD
12:00 - 12:30 Ken Goldberg [In person] - AI + GOFE for Manipulating 1D, 2D, and 3D Deformable Objects
12:30 - 13:30 Lunch Break
13:30 - 14:00 Zackory Erickson [Remote] - TBD
14:00 - 14:30 Chuang Gan [In person] - TBD
14:30 - 15:15 Spotlight Talks #2 (TBA)
  • TBA
15:15 - 16:00 Coffee Break + Poster Presentation
16:00 - 16:30 Yunzhu Li [In person] - Learning Structured World Models From and For Physical Interactions
16:30 - 17:00 Panel Discussion - Reflections & Future Directions in Deformable Object Manipulation
17:00 - 17:30 Best Abstract Award & Closing Remarks

Call for Papers

We invite participants to submit extended abstracts of 3 pages, with unlimited pages for references and appendices, in the IEEE conference style. Submissions will be reviewed by experts in their respective fields. The accepted abstracts will be made available on the workshop website but will not appear in the official IEEE conference proceedings. Participants are encouraged to submit their recent work on the topics of interest mentioned above.

We will also accept “reflection” contributions from former participants who have attended at least one previous edition of the workshop. This unique submission track focuses on reflecting on prior work, allowing contributors to share how their research has evolved or incorporated new advances from other fields, as well as to propose future directions for the community. Page limit is also 3 pages.

Contributions are encouraged, but are not required, to be original. The review process will be single-blind, meaning the submitted paper does not need to be anonymized.

Accepted extended abstracts will be presented in poster sessions and selected spotlight talks. We will request pre-recorded spotlight talks to ensure a smoother execution of the workshop. However, for each selected contribution, at least one author will be required to be present during the workshop for a live Q&A session.

Abstracts can be submitted through Microsoft CMT: WDOICRA2025.

IEEE RAS Computer & Robot Vision workshop award

We are happy to announce the WDO Best Abstract Award sponsored by the IEEE RAS Technical Committee Computer & Robot Vision. The selected contribution will receive a prize of worth 300$. Any extended abstract or reflection submitted to the workshop will be automatically considered for the award.

Important Dates: (dd/mm/yyyy)

  • Submission Deadline: 31/03/2025 (23:59 PST)
  • Notification Date: 16/04/2025 (23:59 PST)
  • Final Submission: 23/04/2025 (23:59 PST)
  • Workshop Date: 23/05/2025

Invited Speakers (alphabetical order):

Jeannette Bohg

Jeannette Bohg


Assistant Professor
Stanford University, USA
Personal website

Talk title: Object-centric or not: How to represent Deformables for Manipulation?

Bio: Jeannette Bohg is an Assistant Professor of Computer Science at Stanford University. She was a group leader at the Autonomous Motion Department (AMD) of the MPI for Intelligent Systems until September 2017. Before joining AMD in January 2012, Jeannette Bohg was a PhD student at the Division of Robotics, Perception and Learning (RPL) at KTH in Stockholm. In her thesis, she proposed novel methods towards multi-modal scene understanding for robotic grasping. She also studied at Chalmers in Gothenburg and at the Technical University in Dresden where she received her Master in Art and Technology and her Diploma in Computer Science, respectively. Her research focuses on perception and learning for autonomous robotic manipulation and grasping. She is specifically interested in developing methods that are goal-directed, real-time and multi-modal such that they can provide meaningful feedback for execution and learning. Jeannette Bohg has received several Early Career and Best Paper awards, most notably the 2019 IEEE Robotics and Automation Society Early Career Award and the 2020 Robotics: Science and Systems Early Career Award.

Yiannis Demiris

Yiannis Demiris


Professor
Imperial College London, UK
Personal website

Talk title: Manipulation of deformable objects in assistive tasks

Bio: Yiannis Demiris is a professor of human-centered robotics at Imperial College London, where he leads the Personal Robotics Laboratory, supported by a Royal Academy of Engineering Chair in Emerging Technologies. His research interests include human-centered multimodal perception, user modelling, and collaborative human-robot control, with a special emphasis on their application to assistive robotics. Prior to joining Imperial, he received his PhD in Intelligent Robotics and his BSc in Artificial Intelligence and Computer Science, both from the University of Edinburgh.

Zackory Erickson

Zackory Erickson


Assistant Professor
Carnegie Mellon University (CMU), USA
Personal website

Talk title: TBD

Bio: Zackory Erickson is an Assistant Professor in The Robotics Institute at Carnegie Mellon University, where he leads the Robotic Caregiving and Human Interaction (RCHI) Lab. His research focuses on developing new robot learning, mobile manipulation, and sensing methods for physical human-robot interaction and healthcare. Zackory’s work spans physical human-robot interaction, healthcare robotics, wearable health sensing, robot learning, physics simulation, multimodal perception, and mobile manipulation. Prior to joining CMU, Zackory received his PhD in Robotics from Georgia Tech with Prof. Charlie Kemp. He also received an M.S. in Computer Science from Georgia Tech and B.S. in Computer Science at the University of Wisconsin–La Crosse. He and his students have received Best Paper Award at HRI 2024, Best Student Paper Award at ICORR 2019, and a Best Paper in Service Robotics finalist at ICRA 2019.

Chuang Gan

Chuang Gan


Assistant Professor
UMass Amherst, USA
Personal website

Talk title: TBD

Bio: Chuang Gan is a faculty member at UMass Amherst and a research manager at the MIT-IBM Watson AI Lab. Previously, Chuang Gan was a postdoctoral researcher at MIT, working with Professors Antonio Torralba, Daniela Rus, and Josh Tenenbaum. Before that, Chuang Gan completed a PhD with the highest honor at Tsinghua University under the supervision of Professor Andrew Chi-Chih Yao. Chuang Gan's research lies at the intersection of computer vision, AI, cognitive science, and robotics. The overarching goal of this research is to develop human-like autonomous agents capable of sensing, reasoning, and acting in the physical world. Chuang Gan's work has been recognized with the Microsoft Fellowship and Baidu Fellowship and has received media coverage from CNN, BBC, The New York Times, WIRED, Forbes, and MIT Technology Review.

Ken Goldberg

Ken Goldberg


Professor, William S. Floyd Jr. Distinguished Chair in Engineering
UC Berkeley, USA
Personal website

Talk title: AI + GOFE for Manipulating 1D, 2D, and 3D Deformable Objects

Bio: Ken Goldberg (IEEE Fellow, 2005) is President of the Robot Learning Foundation and William S. Floyd Distinguished Chair of Engineering at UC Berkeley and Chief Scientist of Ambi Robotics and Jacobi Robotics. Ken leads research in robotics and automation: grasping, manipulation, and learning for applications in warehouses, industry, homes, agriculture, and robot-assisted surgery. He is Professor of IEOR with appointments in EECS and Art Practice. Ken is Chair of the Berkeley AI Research (BAIR) Steering Committee (60 faculty) and is co-founder and Editor-in-Chief emeritus of the IEEE Transactions on Automation Science and Engineering (T-ASE). He has published ten US patents and over 400 refereed papers. He has presented over 600 invited lectures to academic and corporate audiences.

Yunzhu Li

Yunzhu Li


Assistant Professor
Columbia University, USA
Personal website

Talk title: Learning Structured World Models From and For Physical Interactions

Bio: Yunzhu Li is an Assistant Professor of Computer Science at Columbia University. Before joining Columbia, he was an Assistant Professor at UIUC CS, spent time as a Postdoc at Stanford, and earned his PhD from MIT. Yunzhu’s work has been recognized with the Best Paper Award at ICRA, the Best Systems Paper Award, and as a Finalist for the Best Paper Award at CoRL. Yunzhu is also the recipient of the AAAI New Faculty Highlights, the Sony Faculty Innovation Award, the Adobe Research Fellowship, and was selected as the First Place Recipient of the Ernst A. Guillemin Master’s Thesis Award in AI and Decision Making at MIT. His research has been published in top journals and conferences, including Nature and Science, and has been featured by major media outlets.

Shuran Song

Shuran Song


Assistant Professor
Stanford University, USA
Personal website

Talk title: TBD

Bio: Shuran Song is an Assistant Professor of Electrical Engineering at Stanford University. Before joining Stanford, she was faculty at Columbia University. Shuran received her Ph.D. in Computer Science at Princeton University, BEng. at HKUST. Her research interests lie at the intersection of computer vision and robotics. Song’s research has been recognized through several awards, including the Best Paper Awards at RSS’22 and T-RO’20, Best System Paper Awards at CoRL’21, RSS’19, and finalists at RSS, ICRA, CVPR, and IROS. She is also a recipient of the NSF Career Award, Sloan Foundation fellowship as well as research awards from Microsoft, Toyota Research, Google, Amazon, and JP Morgan.

  • Jeannette Bohg, Associate Professor, Assistant University, USA
  • Yiannis Demiris, Professor, Imperial College London, UK
  • Zackory Erickson, AAssistant Professor, Carnegie Mellon University, USA
  • Chuang Gan, Assistant Professor, UMass Amherst, USA
  • Yunzhu Li, Assistant Professor, Columbia University, USA
  • Shuran Song, Assistant Professor, Stanford University, USA

Organizers

  • Alberta Longhini, KTH Royal Institute of Technology, Sweden
  • Michael C. Welle, KTH Royal Institute of Technology, Sweden
  • Martina Lippi, Roma Tre University, Italy
  • Lawrence Yunliang Chen, University of California, Berkeley, USA
  • Daniel Seita, University of Southern California, USA
  • Júlia Borràs Sol, Institut de Robòtica i Informàtica Industrial (CSIC-UPC), Spain
  • Danica Kragic, KTH Royal Institute of Technology, Sweden
  • David Held, Carnegie Mellon University, USA

Contact

If you have any questions please contact Alberta Longhini at the email: albertal AT kth DOT se

Acknowledgment

The Microsoft CMT service was used for managing the peer-reviewing process for this conference. This service was provided for free by Microsoft and they bore all expenses, including costs for Azure cloud services as well as for software development and support.